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REASONS FOR THE PROJECT 

In re::ent years the Texas Department of Human Ser- 
vices (DriS) has faced a rising need for child protective 
services (CPS) that exceeds the Department's capacity for 
timely response to all cases. In 1983, for example, the 
reported incidence of child abuse or neglect rose by 
almost 9 percent over 1982. To complicate the problem, 
this increase came at a time when the available resources 
were shrinking. 

Shrinking resources and increasing caseloads have 
limited the time that workers can give each case and 
made it more difficult for them to make decisions 
efficiently, accurately, and consistently. As a result, 
families may not be offered appropriate services, 
inappropriate foster care placements can occur, and 
children sometimes have to wait longer for a decision 
about placement in an adoptive home. 

To address these problems, DHS in September 1984 
undertook the two-year Case Decision Project. Its goal 
was to improve productivity and impact in child protec- 
tive services by applying automation to parts of the case 
investigation process. 



WHAT THE PROJECT ACCOMPLISHED 
( AND VJHAT REMAINS TO BE DONE) 

The project produced a Case Investigation Decision 
Support System (CIDSS) , which has two parts: 

1. a manual version — a workbook that leads users 

through a standardized method for obtaining case 
information and making case decisions; and 
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2. an automated version. 



Although the workbook (Appendix A) was intended as 
a "stepping stone" toward the automated version, it can 
be used by itself. In a pilot test, the workbook gained 
broad acceptance. Part I of this document gives a full 
report on evaluation of the workbook, and the "Main 
Findings and Conclusions" are summarized on page ix 
(one of the "yellow pages" at the end of this preface) . 

Pilot testing of the automated CIDSS discovered a 
number of problems. One aspect of the system seemed 
useful to pilot participants; the other, under present 
circumstances, did not prove feasible. Part II of this 
docximent gives details on evaluation of the automated 
CIDSS, and "Main Findings and Conclusions" are summar- 
ized on page x. 

In R&D work the discovery of problems can be 
highly useful information. DHS staff are now at work 
refining the automated CIDSS with statewide implementa- 
tion planned for 1988. 

Applying automation to the process of investigating 
cases takes considerable time and effort. However, the 
potential benefits — among which are more uniform col- 
lection of information, more accurate decisions, and 
greater efficiency — appear to justify the costs. By 
sharing its experience through this report, the Depart- 
ment hopes to help others embarking on automation reap 
the benefits sooner than they might have otherwise — and 
at lower costs in time and effort. 



vii 



ERLC 



11 



SUGGESTIONS ON HOW T O USE THIS REPORT 



Readers have differing information needs, and all are 
pressed for time. We (the authors and editor) have tried 
to arrange this report with you in mind. Generally, we 
suggest that you READ THE YELLOW PAGES first. 

Iff yyj have. . • You might first read . . . 



10-15 mins^ Main Findings and Con-- 

elusions (pp . ix-x) 
other Lessons Learned (p. xi) 



20-30 mins. the preceding plus 

Part I Conclusions (1-64 to 1-65) 
Part II Conclusions (11-21 to 1-23) 



30-45 mins. the preceding plus 

Part I Summary of Findings 
(1-62 to 1-63) 
Part II Findings 
(11-18 to 11-20) 

45-60 mins. the preceding plus 

Part II "subsection directories" 
(pp. 1-19, 1-24, 1-39, 1-52) 



To aid this type of "skim reading," a GLOSSARY (located 
just before the back cover) gives definitions of special 
terms and acronyms. 

Although the evaluation was conducted by specialists 
with doctoral-level training, we have tried to make the 
entire report accessible to the general reader. Special- 
ists in evaluation, systems analysis, or programming who 
want more details should contact David Sheets at the 
address on the title page. 
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MAIN FINDINGS AND CONCLUSIONS 



I: CIDSS WORKBOOK 

Attitudes toward The Case Decision Project produced the Case Investigation 

Standardization Decision Support System (CIDSS). The system has two 

parts: (I) a manual workbook and (II) an automated system. 
Each part was evaluated separately. (As a basis for devel- 
oping CIDSS, the project also produced a conceptual model 
of the case investigation process.) 



Evaluation A large majority of surveyed caseworkers and supervisors 

of Workbook thought that a standardized method for gathering case 

information and making case decisions is a good idea. They 
would use the workbook of their own volition. 

Conclusion. The workbook's standardized approach found 
general acceptance. 



Evaluation Seventy percent of those surveyed thought the workbook 

of Training better than previous methods of documenting cases. The 

consensus was that the workbook made it easier to meet 
program standards and did not add to the overall effort of 
investigating a case. Shortcomings cited-the workbook 
isn't well suited to documenting complex cases, and refine- 
ments aie needed to better reflect the actual sequence of 
case investigation. 

Conclusion, Project managers should study the bf^nefits 
of suggested refinements. 



Evaluation Trainees gave instructors and training material positive 

of Pilot Test ratings, though they requested more definitions, examples, 

and follow-up sessions. Scores on exercises that tested 
understanding of the model and proficiency in using the 
workbook indicated a need for improvements in training. 

An interesting finding was that those who scored highest 
on the exercises were likely to have reported positive 
attitudes toward the idea of standardization. 

Conclusion. With more detailed training, written 
instructions for using the workbook, and follow-up sessions 
the workbook can and should be implemented statewide. 

At the end of an eight-week pilot test, most staff had a 
higher opinion of the workbook than they did before the 
tesu 

Conclusion., The workbook is at least an acceptable and 
probably a good alternative to previous methods. 

ix 
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II. AUTOMATED CIDSS 



The automated CIDSS was intended to handle information 
for two purposes: 



Reactions to 
the Pilot Test 



Caseworkers' 
Comments 



Supervisors' 
Comments 



Conclusions 



1. to record case information and analyze it to aid in 
decisions on whether to open or close a case 
(DECISION SUPPORT SYSTEM-DSS); 

2. to receive and assign cases (intake), track their 
status, and produce information to aid management 
of the work (MANAGEMENT INFORMATION SYS- 
TEM-MIS). 

In the pilot test of the automated CIDSS, staff reported 
largely negative experience with the DSS aspect but fairly 
positive reaction to the MIS aspect. The latter offered 
some clear benefits for day-do-day op nations, whereas the 
former seemed unhelpful. 

Caseworkers complained that the automated CIDSS was a 
slow and cumbersome way of recording case information 
compared to taped dictation. They also voiced a need for 
better training and follow-up, a user's guide to the soft- 
ware, and on-site consultation. They felt they needed a 
period of relief from their regular work load to learn the 
new system. And access to terminals at convenient times 
was a problem. 

Supervisors agreed about the shortcomings of training. 
They also felt that data entry was not a good use of case- 
workers' time. Supervisors still think that an automated 
system can prove helpful for assigning and managing cases. 

Although CIDSS was not designed primarily as an MIS, pilot 
test staff found this aspect much more useful than the DSS 
aspect. CIDSS is being redesigned to provide a greater 
range of managment information. Project managers believe 
that experience with an MIS will prepare the ground for 
successful implementation of a DSS. 
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MAIN FINDINGS AND CONCLUSIONS 



I: CIDSS WORKBOOK 



The Case Decision Project produced the Case Investigation 
Decision Support System (CIDSS). The system has two 
parts: (I) a manual workbook and (II) an automated system. 
Each part was evaluated separately. (As a basis for devel- 
oping CIDSS, the project also produced a conceptual model 
of the case investigation process.) 



Attitudes toward 
Standardization 



A large majority of surveyed caseworkers and supervisors 
thought that a standardized method for gathering case 
information and making case decisions is a good idea. They 
would use the workbook of their own volition. 

Conclusion. The workbook*s standardized approach found 
general acceptance. 



Evaluation of 
Workbook 



Seventy percent of those surveyed thought the workbook 
better than previous methods of documenting cases. The 
consensus was that the workbook made it easier to meet 
program standards and did not idd to the overall effort of 
investigating a case. Shortcomings cited-the workbook 
isn*t well suited to documenting complex cases, and refine* 
ments are needed to better reflect the actual sequence of 
case investigation. 

Conclusion. Project managers should study the benefits 
of suggested refinements. 



Evaluation 
of Training 



Trainees gave instructors and training material positive 
ratings, though they requested more definitions, examples, 
and follow-up sessions. Scores on exercises that tested 
understanding of the model and proficiency in using the 
workbook indicated a need for improvements in training. 

An interesting finding was that those who scored highest 
on the exercises were likely to have reported positive 
attitudes toward the idea of standardization. 

Conclusion. With more detailed training, written 
instructions for using the workbook, and followup sessions 
the workbook can and should be implemented statewide. 



Evaluation 
of Pilot Test 



At the end of an eight-week pilot test, most staff had a 
higher opinion of the workbook than they did before the 
test. 

Conclusion. The workbook is at least an acceptable and 
probably a good alternative to previous methods. 
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PART I 
Impact Evaluation 

CASE INVESTIGATION DECISION SUPPORT SYSTEM (CIDSS) 

MANUAL WORKBOOK 



EXECUTIVE SUMMARY 



The Protective Services for Families and Children 
(PSFC) Branch is developing automated support for 
child protective services (CPS) field staff. The case 
investigation workbook began as a recording instrument 
to support software intended to automate CPS case 
investigation reporting and recording. The workbook 
was a prototype of the software system and was used by 
CPS specialists to test the planned content and organi- 
zation of the software. During prototyping, it became 
apparent that the workbook could stand on its own, and 
that it had advantages over current procedures for 
recording case investigations. 

The pilot test of the CIDSS workbook was in- 
tended to help refine the content and organiza- 
tion of the software and to test the usefulness 
of the workbook without softwere. 

SOFTWARE IMPLICATIONS 

The format of the workbook was generally accept- 
able. Staff did not suggest revisions that influence 
the content or organization of the software. Staff 
comments recommend a high level of training and support 
during workbook implementation. An implication of 
their comments is that — 

the workbook and the software should be imple- 
mented separately to minimize the likelihood of 
training overload. Results of the training 
evaluation are consistent with this conclusion. 
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WORKBOOK IMPLICATIONS 



The pilot test staff had generally positive reac- 
tions to the workbook. Nearly two-thirds of them said 
that they would probably or definitely continue to use 
the workbook if it were their decision to make. 

Over three-quarters of the respondents agree 
with PSFC that the workbook is a good idea. 



PROBLEMS AND SOLUTIONS 

A few problems were identified. The solutions to 
most of the problems consisted of better directions for 
using the workbook and more training, especially prac- 
tice and examples. Frequently, training needs were 
specified as a "need for follow-up training." It is 
appropriate to conclude that — 

given (1) detailed training, (2) written direc- 
tions on use of the workbook and (3) training 
follow-up at one to two months, the workbook 
can nnd should be implemented statewide. 
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1. BACKGROUND 

As a convenience to the reader, sections (and some 
of the longer subsections) are preceded by a "mini" 
table of contents dubbed a "directory." 

Section 1 Directory 

1 . 1 Introduction ( I-l ) 

1.2 CPS Automation Plan (1-4) 

1.3 Developing the Worlcbook (1-7) 



1.1 INTRODUCTION 

CPS Automation Stages . The Protective Services for 
Fcimilies and Children (PSFC) Branch of the Texas De- 
partment of Hximan Services (OHS) is engaged in an 
effort to automate child protective services (CPS) . 
This effort is being pursued in three stages. 

o The first stage, providing automated support for 
the intake ftinction, has been completed. 

o The second stage, automated support for case 
investigations, has been pilot-tested and is 
currently being redesigned for statewide imple- 
mentation. 

o The third stage, automated support for planning 
and managing cases that are opened for services, 
is the most complex and sensitive of the three 
efforts and is still under development. 

The Case Decision Project contributed to the second 
stage of CPS C:utomation. 

I-l 
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The Case Decision Project developed the Case 
Investigation Decision Support System (CIDSS), 
which consists of two parts: 

1. The CIDSS manual workbook (Part I of this 
report describes its evaluation and reports 
on the support needed to facilitate intro- 
duction of the workbook as Texas' standard 
form for recording CPS investigations) . 

2. The CIDSS automated system, a software 
version of the workbook (Part II of this 
report describes its evaluation, reports on 
problems of introducing an automated system, 
and suggests some solutions). 



The Case Investigation Decision Supporr System 
(CIDDS) manual workbook began as a recording instrument 
and guide to support software intended to automate CPS 
case investigation reporting and recording. The work- 
book was a paper representation of the software system. 
The software and the workbook were based on a model 
that increases objectivity by separating the investiga- 
tion and decision*-making aspects of the process (see 
subsection 1.3). 

Goals for the Workboo k. The workbook was treated 
as a prototype of the software system and used by case 
investigation specialists V.o test the planned content 
and organization of the software. During prototyping, 
it became apparent that the worlcbook could stand on its 
own and that it had advantages over current procedures 
for recording case investigations. The pilot test of 
the CIDSS workbook was intended to help refine the 
content and organization of the software and to test 
the usefulness of the workbook without software. The 
decision to test the feasibility of statewide implemen- 

1-2 
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tation without software was made to see whether the 
workbook could help CPS achieve three goals: 

o Standardize recording of case inve stigations . 
Use of the workbook provides a standardized 
format for recording and reporting the results 
of case investigations • Standardization facili- 
tates communication and provides a basis for 
identifying the completeness and adequacy of the 
case record. 

o Achieve standardization quickly . The automated 
system will take years to implement statewide, 
and it is likely that rural areas are many years 
away from access to networked automated record 
keeping. 

o Separate learning investigation strategies from 
learning to use a computer . Using the workbook 
before using the software separates learning the 
standardized investigation system from learning 
to use the software and spreads the effort 
required over a longer period of time. 

Early in the development of the automated system, 
versions of the workbook were used to prototype the 
form ^nd content of the automated system. In July of 
1986 the basic design of the software and thus the 
workbook had been established. Between August and 
November the workbook was refined, training designed, 
and the CIDSS workbook pilot test planned. The pilot 
test included nearly 100 CPS investigation specialists 
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from two DHS regions and ran for four months, November 
1985 through Febru^.ry 1986* 

For the past several years human service agencies 
have found themselves in a situation of declining re- 
sources and expanding client service i.aeds. One way to 
address this problem is to reduce tiie cost of service 
delivery and improve the effectiveness of services de- 
livered. Computers have been successful m helping 
private industry reduce overhead and lower unit costs. 
DHS, like many other public agencies, has invested in 
automation in the hope of replicating private industry 
success. Figure 1-1 shows the proposed flow of CPS 
automation systems and indicates the stages completed 
so far. 

Needs the System Must Meet . CPS's automation goals 
include development of an information system that meets 
a broad range of needs for three categories of staff. 

1. Case Investigation Specialists ! for the spe- 
cialist in the field, needs include — 

o a standardized format for recording; 

o quick retrieval, revision, and t;ransfer of 

case information; 
o automated production and transmission of 

required forms; 
o support in meeting program requirements; 
o case decision support systems; 
o work load and caseload status summaries; and 

2. Supervisors ; for the supervisor, system needs 
include — 
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Figure I-l. Proposed flow of CPS Automated Systems 
and their Stages of Development 



Local Sysceffls 



.lis 
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o more easily understood and interpreted 
records ; 

o automated systems to facilitate work load 

management ; and 
o unit work load and caseload status summaries 

3. Regional and state staff : for regional and 
state program staff, system needs include — 
o vork load and performance summaries by pro- 
gram director, rec/ion, and branch; 
o client population descriptions and tracking 
ability; 

o management decision support; and 

o policy testing (modeling) capability. 

How CPS Automated Systems Aid Casework. In a local 
CPS office, the pi:oposed CPS systems will perform three 
functions (also partially represented in the middle of 
figure I-l. The first function, Automated ^PPERl 
Intake (AMI) , employs software to automate and provide 
printing and telecommunications capability for CPS 
intake. (The precursor to AMI was the Prompted Intake 
System.) Software to automate intake has been pilot 
tested and is currently in use by one intake site. As 
telecommunications and computer facilities become 
available, AMI will be implemented in additional sites. 

The second function, the Cage Investigation Deci- 
sion Support System (CIDSS) , has the following com- 
ponents : 

1. the CIDSS workbook; 

2. the automated CIDSS, which features — 
o automated case recording, 



^Maintaining, Preparing, and Producing Executive 
Reports — a UNlSYS-developed software language. 
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o automation of case management tasks, and 
o automated management information reporting. 

The CIDSS Workbook has been pilot tested and is the 
subject of Part I in this eve.luation report • Case re- 
cording software has been tested in the field. The 
decision support system is under development. 

The third set of software functions, in the initial 
stage of design, will automate recording related to 
case management and planning. 



1.3 DEVELOPING THE WORKBOOK 

As a convenience to the reader, lengthier subsec- 
tions such as this are provided with a directory. 

Subsection 1.3 Directory 

1.3.1 Introduction to CIDSS Development (1-7) 

1.3.2 Specifying the Data Elements (1-8) 

1.3.3 Developing the Model (1*9) 

1.3.4 Finalizing the Instrument (I-ll) 



1.3.1 Introduction to CIDSS Development 

At the beginning of the project the essential tasks 
were the following: 

o specify the data elements that need to be col- 
lected during an investigation in order to 
arrive at a sound decision about case disposi- 
tion; 
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o develop a model of the decision-making process; 
and 

o incorporate these two factors into an instrument 
that allows for ease of data collection by the 
worker and ease of case reading by the super- 
^ visor. 

1.3.2 Specif vinq the Data Elements 

The first step was to design a manual investigation 
workbook. This workbook was submitted to a process of 
review and modification by a work group made up of CPS 
experts from across the state. The group included 
field staff, CPS supervisors and administrators, and 
project managers. This combination of experience and 
perspectives ensured that the initial versions of the 
workbook were based on field experience, management 
analyses, and an extensive review of the clinical and 
research literature. After producing three revisions 
of the workbook, the group felt that no further devel- 
opment could take place without testing in actual work 
settings. 

As a second step, the work group elected to carry 
out limited and informal field tests of the draft CIDSS 
workbook. A field-ready version of the workbook was 
produced, and three sites in Texas volunteered to test 
it for 60 days. Staff expected to use the workbook re- 
ceived an overview of its purpose and origin. They 
used the workbook for 30 days, and their recommenda- 
tions for modification were used for revisions. The 
revised workbook was then introduced to the same sites, 
and after 30 to 45 days of use the staff's reactions 
and recommendations were solicited and used in revising 
the workbook again. 

1-8 

ERiC 2C 



The testing and revision resulted in the pilot test 
version of the CIDSS workbook. The process also led to 
identification and specification of the CPS investiga- 
tion decision-making process. Understanding the decis- 
ion-making process allowed project managers to develop 
a model of the process and to design training for pilot 
test staff. 

1.3.3 Developing the Model 

As the workbook was being field tested, project 
managers were working on specifying training needs for 
a pilot test. A key element of developing clear and 
effective training was specification of a decision 
model, consistent with CPS literature, that could be 
applied in the field. A review of the literature 
indicated that the investigation consists of two sepa- 
rate decision processes: (1) assessing potential risk 
to the child and (2) assessing the capacity of the 
family and other resources to address the abuse situa- 
tion. 

Assessing risk included activities related to 
judging whether child abuse had occurred and the like- 
lihood of future abuse. If investigators found that 
abuse was likely to occur in the future they proceeded 
to the second process, an assessment of what types of 
intervention would alter the abusive circumstances to 
minimize risk of future abuse. 

Throughout the field tests, staff were asked to 
describe their use of and reactions to the workbook. 
Their reactions supported the separation of the deci- 
sion processes but indicated that frequently the pro- 
cesses overlapped or were carried out simultaneously. 
The consistency among findings from the literature 
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review and interviews with field staff led project 
staff to accept the separation of decision processes 
and base the investigation model on this feature of 
investigations • 

Feedback from staff and the opinions of the work 
group had resulted in a workbook with sections that 
addressed elements of each decision process. Analysis 
of the workbook sections and staff reports of how they 
used each section and subsection helped identify the 
activities included in the two investigation processes. 
The activities and processes were identified as falling 
into two stages. 

Stage 1; Assessing Risk . Data from three general 
areas are collected and analyzed to determine the in- 
tensity of risk for abuse/neglect to the child • 

o The Event ; Did the alleged abuse/neglect occur? 

o The Effect : How severe was the abuse/neglect, 
and what were its effects upon the child and 
family? 

o The Environment ; To what extent does the psy- 
chosocial and physical environment act to sup- 
port or prevent the occurrence of abuse/neglect? 

If some degree of risk intensity is found, Stage 2 of 
the model is invoked. 

Stage 2: Assessing Available Intervention/Treat- 
ment Resources . Resources available to reduce the risk 
intensity are assessed in order to arrive at one of two 
case decisions; (1) close the case, or (2) open the 
case for in-home services or removal of the child from 
the home. Three types of resources are assessed. 
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o The Family : Does the nuclear and/or extended 
family have sufficient resources to reduce the 
risk intensity? If so the case can be closed. 

o The Community ; Are community resources avail- 
able and accessible to the family to reduce the 
risk intensity? If so, the case can be closed 
after appropriate referrals are made. 

Child Protective Services : If the child is 
still at risk after the application of famixy 
and community resources, the family is eli^^ible 
for child protective services. The level of 
risk at this point will determine the level of 
intervention (in-*home services or removing the 
child) . 

The model based on this analysis of the investiga- 
tion process is called the RIF-RAF Model (Risk Inten- 
sity Factors/Resource Availability Factors) . Figure 
1-2 attempts to present it visually. 

1.3.4 Finalizing the Instrument 

The workbook was introducted to a new site with 
training. Input from staff at the new site was pre- 
sented at the next work group meeting. During the 
meeting another revision of the workbook was produced. 
At this meeting the work group decided that the work- 
book was close to its final form, that the training 
plan was adequate, and that a formal pilot test of the 
workbook should be carried out. Evaluation up to this 
point had been informal, and it was felt that a more 
reliable evaluation was needed to provide a solid 
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foundation for system of this importance and com- 
plexity. 



/ 



Figure 1-2. RIF/FAF Model of CPS Case Investigations 
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The RIF/RAF Model (and the analysis from which it 
came) provided the basis for training the pilot test 
staff. 
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2, METHODS 



-Section 2 Directory- 



2 . 1 Introduction (p. 1-13) 

2.2 Population and Sample (p, 1-13) 

2.3 Evaluation Research Design (p. 1-14) 

2.4 Types of Data Analysis (p. 1-16) 



2,1 INTRODUCTION 

The methods section includes three subsections. 
First, the population and ssunple are discussed in terms 
of their adequacy to answer the research questions. 
Second, the research design and the research questions 
are discussed. Third, the data analyses are intro- 
duced. (The resulting findings are presented in 
section 3 . ) 



2,2 POPULATION AND SAMPLE 

The work group for the pilot wanted to ensure that 
the workbook was tested with all types of cases as well 
as in both urban and rural settings. With these 
criteria expressed, two regions volunteered as pilot 
sites: Region 11 (Houston) and Region 8 (Corpus 
Christi and the Rio Grande Valley) . These sites met 
the selection criteria and were chosen for the pilot. 
Participation within each region was also voluntary, 
and 117 staff volunteered to test the workbook. Table 
I--1 shows the breakdown of specialist and supervisor 
staff by region. 
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TABLE I-l 

Pilot Test Volunteers by DHS Region and Job 



Staff Position Region 8 Region 11 

Program Directors 3 3 

Unit Supervisors 12 3 

Case Specialists 71 25 

Total 86 31 



The voluntaiY nature of selection may be a bias in 
the sample. However^ volunteering was done at a man- 
agement level, and individual workers were required to 
participate if their supervisors opted to do so. Also, 
nearly all staff managed by the six project directors 
used the workbook. 

2.3 EVALUATION RESEARCH DESIGN 

2.3.1 Evaluation Questions 

The CIDSS workbook pilot test was designed to 
answer the following general evaluation questions: 

o How do workers respond to use of the workbook? 

o What are the problems with the layout or content 
of the workbook? 

o What types of training and support are necessary 
to support introduction and use of the worJcbook? 

The evaluation also set out to identify what implica- 
tions the workbook evaluation might have for the 
content and organization of the CIDSS software. 
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2.3.2 Evaluation Strategies 



The evaluation plan included three strategies to 
answer these questions: (1) questionnaires, (2) tele- 
phone contacts, and (3) individual and group discus- 
sions. 

Questionnaires . There were two types of cjuestion- 
naires. 

o One type, administered to both case inves- 
tigation specialists and to supervisors, was a 
* series of attitude surveys (Appendix A) . These 
surveys asked staff about their experience with 
the workbook, their reactions to standardization 
of case investigation record keeping, and their 
opinion on aspects of the workbook and their 
job. The attitude questionnaires were adminis- 
tered before the pilot test, aJter staff 
training, and at the end of the pilot. The 
questionnaire administered after training 
included both open- and closed-ended questions 
evaluating the training. 

The other type of questionnaire was the Im- 
plementation Factors (IF) Survey (Appendix B) . This 
survey is a simple procedure for anticipating implemen- 
tation problems and monitoring problem resolution. 
Supervisors and program directors in the pilot sites 
completed IF surveys b^.fore, during, and at the end of 
the pilot. 

Telephone Contacts . The second evaluation strategy 
was a telephone contact system that allowed each worker 
quick access to problem resolution. The system was set 
up to help identify workbook problems quickly and to 
provide a means of responding as quickly as possible 
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not only to the case investigation specialist who 
identified the problem but to all pilot test partici- 
pants. Those with workbook problems had the option of 
contacting the project manager directly or contacting 
their supervisor to relay a problem to the project 
manager, who responded directly to the caller identify- 
ing a problem. The project manager then had the option 
of using a telephone tree or a memo (depending on the 
urgency of the problem) to report the resolution of the 
problem to pilot test staff. 

Individual and Group Discussions . The third 
evaluation strategy consisted of informal contacts and 
group discussions organized around complaints and 
problems. Several of these contacts and discussions 
were carried out before the pilot test began. DHS 
travel restrictions and the small nximber of problems 
that came up meant there were few informal contacts and 
only two discussions with groups of staff during the 
pilot. 



2.4 TYPES O F DATA ANALYSIS 

The data analyses include frequency, breakdowns, 
and crosstabulations. Frequencies are used to display 
response rates by response category. Breakdown 
analyses are used to show mean responses across 
respondents. Crosstabulations are used to indicate 
associations among responses to different questions. 
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3 , FINDINGS 

Section Directory 

3 . 1 Introduction (1-18) 

3.2 Attitudes toward Work and Standardization 
(1-19) 

3.3 Evaluation of the Workbook (1-24) 

3.4 Evaluation of Training (1-39) 

3.5 Evaluation of the Pilot Test (1-52) 

3.6 Summary of Overall Findings (1-62) 
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3,1 INTRODUCTION 



The evaluation findings are organized into five 
subsections. Subsection 3.2, "Attitudes toward Work 
and Standardization," provides a baseline measure of 
participants* attitudes about their jobs and the work 
they do. The subsection also presents information 
concerning how they expect standardization will 
influence the way they carry out a case investigation. 

Subsection 3.3, "Evaluation of the Workbook," 
presents findings concerning how attitudes toward 
standardization and opinions of the workbook changed 
over the course of the pilot. Findings are presented 
on strengths and weaknesses of the workbook. The last 
part of the subsection identifies pilot test staff 
suggestions for improving the workbook. 

Subsection 3.4, "Evaluation of Training," includes 
the findings from two post-test exercises, workers' 
reactions to the instructor and the training content, 
and a critique of the training content. This sxibsec- 
tion includes an analysis of the relationship of 
performance on the post-training exercises and re- 
sponses to two types of workbook evaluation questions. 

Subsection 3.5, "Evaluation of the Pilot Test," 
reports on findings from the Implementation Factors 
Questionnaire. The questionnaire tracked supervisors* 
assessments of the workbook and their opinions of pilot 
test operations over time. 

Subsection 3.6 summarizes the main findings 
described in subsections 3.2 through 3.5. 
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3.2 ATTITUDES TOWAR D WORK AND STANDARDIZATION 

Subsection 3.2 Directory 

3.2.1 Data Sources (1-20) 

3.2.2 Attitudes toward Job — Generally Positive, 
Little Change (1-20) 

3.2.3 Attitudes toward Standardization — Little 
Change Overall (1-21) 

3.2.4 Summary — Attitudes Little Influenced 
(1-23) 
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3.2.1 DAT A SOURCES 



Pilot test staff were asked about eight aspects of 
their work, before and at the end of the pilot test. 
The questions before the pilot were asi^ed before staff 
were introduced to the workbook. The questions were 
asked again at the end of the *)ilot to see whether use 
of the workbook had an influence on staff attitudes 
toward work. The question response scales ranged from 
117 II to "1" with "7" representing the most desirable 
attitudes toward work and "1** the least desirable. 



2.2.2 Attitudes toward Job-- 

Generally Positive, Little Change 

Examination of the average scores shows little 
change in attitudes from before the pilot to after 
using the wbrkbook for four months (table 1-2) . 

The questionnaire item read **Here are some words 
and phrases which can be used to describe how you see 
your job. For example, if you think your job is very 
•boring* circle number 1, right next to the word 
•boring.* If you think you job is very • interesting, • 
circle number 7, right next to the word 'interesting.' 
If you think it is somewhere in between, circle a 
number between 1 and 7 . " 

Staff reported generally positive attitudes toward 
tho eight aspects of their jobs. The highest scores 
indicated that the pilot staff find their job "Inter- 
esting" (6.1) and "Worthwhile" (6.2). The lowest 
average score was 4.2 on the Hopeful vs. Discouraging 
scale at the end of the pilot. Since 4.0 is the mid- 
point of the scale, 4.2 indicates a slight tendency for 
staff to respond that they find their work "Hopeful." 

1-20 



38 



TABLE 1-2 
Attitudes toward Job 



Attitude Categories 



Before 
Pilot 



End of 
Pilot 



Interesting vs. Boring 
EnjoyeUsle vs. Miserable 
Worthwhile vs. Useless 
Friendly vs. Lonely 
Full vs. Empty 
Hopeful vs. Discouraging 
Rewarding vs. Disappointing 
Brings out the best In me 

vs. doesn't give ne 

a chance 



6.1 
4.4 
6.2 
4.3 
5.2 
4.4 
4.6 



4.8 



6.1 
4.6 
6.2 
4.7 
5.6 
4.2 
4.8 



4.8 



Average 



5.0 



5.1 



The average scores of 5.0 and 5.1 Indicate that before 
and after the pilot test staff had good attitudes 
toward their jobs and the work they do. 

3.2.3 Attitudes toward Standardization- 
Little Change Overall 

There were 11 questions on staff opinions concern- 
ing standardization and the workbook. These questions 
were asked before the pilot, after training on how to 
use the workbook, and at the end ox the pilot test. 
There was very little difference in staff opinions 
concerning the workbook at each of the three times. 
Their overall mean score changed from 4.4 before the 
pilot to 4.3 after training to 4.2 at the end of the 
pilot (table 1-3) . 
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TABLE 1-3 

Attitudes toward Standardization and the Wor)cbook 



Attitude Statements^ Before After After 

Pilot Training Pilot 



1. 


Not Increase my work load' 


4 


. 6 


4 • 


2 


4 • 


4 


2 . 


Increase freedom on job 


3 


. 7 


3 . 


5 


3 . 


4 


3 . 


Improve quality of 
















work I produce 


4 


. 7 


4 . 


6 


4 . 


0 


4. 


Not more di-flcult to 
















meat deadline 


4 


.8 


4. 


5 


4. 


7 


5. 


Not make It difficult 
















to do good job 


5 


.2 


5. 


0 


5. 


0 


6. 


Make work more challenging 


3 


.8 


4. 


0 


3. 


5 


7. 


Not make work more frustrating 


4 


.6 


4. 


4 


4. 


3 


8. 


Not decrease my discretion 
















on job 


4 


.2 


4. 


4 


4. 


7 


9. 


Increase ability to 
















get work done 


4 


3 


4. 


3 


4. 


3 


10. 


Easier to keep up 
















with work load 


4 


.S 


4. 


3 


4. 


1 


11. 


Make my job more Interesting 


4 


.1 


3. 


9 


3. 


6 




Average 


4 


.4 


4. 


3 


4. 


2 



1. The Introduction to the 11 statements read as 
follows: "We are Interested In learning how the 
CIDSS Investigation guide has changed your job and 
the work you do. A number of possible changes are 
listed below. Please indicate your level of 
agreement or disagreement with each statement by 
checking the appropriate numbered box on the seven- 
point scale." 

2. These questions were asked with six por.itive 
statements and five negative ones. For this table, 
"not" was added to the negative statements to make 
them positive, and scores for these questions were 
converted to provide comparable scores across 
statements. 
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Mean ratings for three of the statements (numbers 
3, 8, and 10) show a notable change from before train- 
ing to after training. The scores that dropped (state- 
ments 3 and 10) indicate that experience with the form 
did not meet staff expectations to "improve the quality 
of work" and for making it "easier to keep up with work 
load." The score that increased (statement 8) indi- 
cated that use of the workbook did not decrease spe- 
cialists' and supervisors' "discretion on the job" as 
much as they had anticipated. The average scores from 
before the pilot, after training, and after the pilot 
test indicate that in general there was little change 
in staff attitudes toward standardization. 

3.2.4 Summary; Attitudes toward Work 
and Standardization — Little Influenced 

The pilot test of the workbook does not appear to 
have influenced staff attitudes toward work. Overall 
staff responses to the idea of standardization and to 
the workbook as the method of standardization also 
remained fairly steady for the duration of the pilot. 
It would appear that the workbook has had little 
influence on attitudes toward work and standardization. 
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3.3 EVALUATION OF THE WORKBOOK 



Subsection 3.3 Directory 

Opinions of the Workbook 

3.3.1 Majority Would Use the Workbook (1-25) 

3.3.2 Most Favor the Idea 

of Standardization (1-26) 

3.3.3 Workbook Met Most 

Users * Expectations ( 1-27 ) 

Experience with the Workbook 

3.3.4 Plurality Think Case Documentation 
Improved (1-28) 

3.3.5 Usefulness of the Workbook (1-29) 

o Most useful for "quickie" and typ- 
ical cases; least useful for com- 
plex cases (1-29) 

o Most respondents complete workbook 
after the investigation (1-30) 

3.3.6 Effect on Case Investigation Effort (1-32) 

o Workbook took more time for a large 
proportion of respondents (1-32) 

o Easier to meet program standards (1-33) 

o Number of case contacts 
stayed the same (1-3 3) 

o Summary — nc change overall (1-34) 



3.3.7 Open-Ended Comments and Suggestions 
for Improvement (1-3 5) 

o Minor changes (1-3 5) 
o Major changes (1-3 6) 
o General reactions (1-37) 



3.3.9 Summary — Reactions tr Workbook, Though 
Mixed, Tended toward Positive (1-38) 
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Findings concerning the workbook are taken from the 
three surveys carried out over the course of the pilot 
test* Two kinds of data were collected. Quantitative 
data were collected concerning opinions of and reac- 
tions to use of the workbook. Each survey also 
included required and/or optional comment sections. 



3.3.1 Majority Would Use the Workbook 



The key finding concerning the workbook is that 63% 
of the respondents reported that they probably or def- 
initely would use the workbook (table 1-4). 



TABLE 1-4 
Using the Workbook — Pro and Con 

Responses to the question 
"If it were entirely your choice 
would you continue to use the CIDSS workbook?" 



Response Categories Number of Number of Percent- 
Specialists Supervisors ages 



Definitely No 
Probably No 
Not Sure 
Probably Yes 
Definitely Yes 



5 

13 
10 
28 
19 



18% No 



2 
1 
2 
4 



63% Yes 



The question does not address whether or not 
respondents would dislike being required to use it. 
But only 18% reported that they would probably or 
definitely not use the workbook if it were their 
choice. 
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3*3*2 Most Favor the Idea of Standardization 

On three occasions, pilot test staff were asked 
whether a standardized workbook was a good idea: before 
training, after training, and at the end of the pilot. 
Most of the respondents did not see the workbook before 
training, so they were asked whether a standardized 
investigation guide was a good idea for CPS. After 
training and after the pilot, staff were asked whether 
the CIDSS workbook was good idea for CPS. Before 
training 75.7% of respondents indicated that a stand- 
ardized guide was probedDly or definitely a good idea 
(table 1-5) . 



TABLE 1-5 

Opinions on the Idea of Standardization 

Reactions to the question "do you think that 
the use of a standardized investigation guide 
is a good idea for child protective services?" 



Response Before After After 

Category Training Training Pilot Test 





No. 


% 


No. 


% 


No. 


% 


Definitely No 


0 




0 




0 




Probably No 


2 


2.4% 


3 


3.5% 


6 


7.1% 


Not Sure 


18 


21.4% 


23 


26.7% 


13 


15.3% 


Probably Yes 


36 


42.4% 


44 


51.2% 


39 


45.9% 


Definitely Yes 


28 


33.3% 


16 


18.6% 


27 


31.8% 


Total % Yes 




75.7% 




69.8% 




77.7% 



After being trained in use of the CIDSS workbook, 
staff were somewhat less optimistic, with 69.8% report- 
ing they probably or definitely felt that the workbook 
was a good idea. Most notably, the percentage of 
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"Definitely Yes" responses dropped from 33.3% to only 
18.6%, After four months of use 11.1% of respondents 
reported that the CIDSS workbook was probably or 
definitely a good idea. Also, the percentage of 
"Definitely Yes" responses was up to 31.8%, only one 
person different from the prepilot finding of 33.3%. 

3.3.3 Workbook Met Most Users' Expectations 

An additional question was asked to expand upon the 
opinions presented in subsections 3.3.1 and 3.3.2 and 
to ensure comparedDility of findings edDout the idea of 
standardization vs. the CIDSS workbook. After train- 
ing, staff were asked whether the CIDSS workbook met 
their expectations. Nearly half of the trainees 
(48.8%) reported that the workbook did meet their 
expectations (table 1-6). 

TABLE 1-6 

CIDSS Workbook: Realization vs. Expectations 



Responses to the 


statement "The CIDSS 


workbook 


is 


than expected. 


tt 


Response Category 


Respondents 


Percent 


Much Better 


12 


14.0 


Better 


25 


29.1 


As Expected 


42 


48.8 


Worse 


7 


8.1 


Much Worse 


0 





The CIDSS workbook failed to meet the expectations 
of only 8.1% of the trainees. The fact that 91.8% 
reported that the workbook at least met their expecta- 
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tions supports the earlier finding of little if any 
shift in responses from pretraining opinions on the 
idea of standardization to the post-training evaluation 
of the CIDSS workbook. In other words, the workbook 
was as good or better than what staff expected of a 
standardized guide. The only notable difference is 
that post-training responses tended to be more conser- 
vative, with a larger portion of responses in the 
"Probably Yes" category. 



3>3.4 Plurality Think Case Documentation Improved 

Pilot test staff were asked to compare the workbook 
to previous methods of recording by agreeing or dis- 
agreeing with threfc statements: 

o (Using the workbook has) improved the documenta- 
tion in case records. 

o A workbook record is less clear than a record 
before the pilot test. 

o The workbook record makes it hard to really 
understand the case. 

For this report the second and third statements are 
converted to positive statements and the response cate- 
gories are receded to standardize higher nuiabers as 
more desirable responses. 

Table 1-7 shows that 58.3% of the respondents 
agreed that use of the workbook improved documentation. 
Nearly half (45.1%) of the respondents reported that a 
workbook record was not less clear than previous 
investigation records, and 50.6% said that an investi- 
gation recorded in the workbook was not hard to 
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TABLE 1-7 

Workbook's Effect on Documenting Case Investigation 



Response Improved Not Less Not Hard To 

Category Docximentation Clear Understand 

No, % No, % No, % 



Disagree Strongly 


7 


8 


.3% 


4 


4 


.9% 


6 


7. 


2% 


Disagree 


9 


10 


.7% 


10 


12 


.2% 


8 


9. 


6% 


Disagree Somewhat 


5 


6 


.0% 


11 


x3 


.4% 


15 


18. 


1% 


Neither 


14 


16 


.7% 


20 


24 


.4% 


12 


14. 


5% 


Agree Somewhat 


23 


27 


.4% 


15 


18 


.3% 


13 


15. 


7% 


Agree 


20 


23 


.8% 


20 


24 


.4% 


26 


31. 


3% 


Agree Strongly 


6 


7 


.1% 


2 


2 


.4% 


2 


3. 


6% 


Total % Agree 




58 


.3% 




45 


.1% 




50. 


6% 



understand. On the other hand 16.7% felt that the 
workbook did not improve docximentation 25.6% said 
workbook recordings ejre less clear and 27.7% reported 
that the workbook records are hard to understand. 

3.3.5 Usefulness of the Workbook 

Most Useful for "Quickie" and Typical Cases; Least 
Useful for Complex Cases . Three survey questions asked 
whether the workbook was especially good for recording 
(1) "quickie," (2) complex, and (3) typical cases. The 
respondents tended to agree that the workbook was 
especially good for recording quickie and typical 
cases. Table 1-8 shows that 47.7% disagreed with the 
statement "The workbook is especially good foi record- 
ing complex cases." 

1-29 



ERIC 



47 



TABLE 1-8 

Type of Cases Workbook is Useful For 

Responses to the statement "The workbook is 
especially good for ..." 



Response Quickie Complex Typical 

Category No. % No. % No. % 



Disagree Strongly 


7 


8 


.3% 


4 


4.8% 


1 


1.2% 


Disagree 


9 


10 


.7% 


21 


25.0% 


2 


2.4% 


Disagree Somewhat 


9 


10 


.7% 


15 


17.9% 


4 


4.8% 


Neither 


8 


9 


.5% 


11 


13.1% 


14 


16.7% 


Agree Somewhat 


11 


13 


.1% 


12 


14.3% 


25 


29.8% 


Agree 


26 


31 


.0% 


15 


17.9% 


30 


35.7% 


Agree Strongly 


14 


16 


.7% 


6 


7.1% 


8 


9.5% 


Total % Agree 




60 


.8% 




39.3% 




75.0% 



Respondent comments identify two possible explana- 
tions for the disagreement. First, several comments 
and discussions indicated that cases involving a lot of 
people, especially cases with several victims and/or 
perpetrators, required so many pages that it was 
difficult to keep track of case relationships and keep 
the workbook properly organized. Second, in complex 
cases, the workbook space allocation is inadequate. 
For example, recording the text of explanations for 
several allegations requires additional pages. 

Most Respondents Complete Workbook after the 
Investigation . Two survey questions asked how the 
workbook was used in an investigation. Table 1-9 shows 
that 39% of the respondents agree with the statement "I 
refer to the workbook as I do the investigation." 
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TABLE 1-9 
When Respondents Use the Workbook — 
during or after Investigation 



Response During After 

Investigation Investigation 

Largely Completed 



Category 


No. 


% 


No. 


% 


Disagree Strongly 


3 


3.7% 


0 


— 


Disagree 


15 


18.3% 


3 


3.7% 


Disagree Somewhat 


6 


7.3% 


6 


7.3% 


Neither 


26 


31.7% 


2 


25.6% 


Agree Somewhat 


19 


23.2% 


12 


14.6% 


Agree 


11 


13.4% 


36 


43.9% 


Agree Strongly 


2 


2.4% 


4 


4.9% 


Total % Agree 




39.0% 




63.4% 



The table also shows that at least 29.3% of the 
respondents do not use the workbook as a guide during 
the investigation. It is possible that some staff who 
responded "Neither" do not refer to the workbook. 

Only 11% of the respondents disagreed with the 
statement "I fill out the workbook after the investiga- 
tion is substantially completed." Most of the respon- 
dents (63.4%) agreed that they complete the workbook 
after the investigation. 
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3>3,6 Effect on Case Invpstiaation Effort 



The responses to three statements address the 
evaluation question "Does use of the workbook change 
the level of effort required to investigate a case?" 

Workbook Took More Time for a L arge Proportion of 
Respondents , Responses to the statement that use of 
the workbook has not "increased time to complete docu* 
mentation" show that 47.0% of the respondents dis- 
agreed. They reported that the workbook took longer to 
complete (table I-IO) . 



TABLE I-IO 

Workbook's Effect on Case Documentation Time 

Responses to the Statement 
"Use of the workbook has not 
"iricreased time to complete case docuiaentation. " 



Response Category 


Number 


Percent 


Disagree Strongly 


8 


.6% 


Disagree 


20 


.1% 


Disagree Somewhat 


11 


13.3% 


Neither 


20 


24.1% 


Agree Somewhat 


8 


9.6% 


Agree 


13 


15.7% 


Agree Strongly 


2 


3.6% 


Total % Agree 




28.9% 
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Only 28.9% of the respondents indicated that the 
workbook did not take longer to complete than previous 
recording forms, and 24.1% neither agreed nor dis- 
agreed. In other words, when using the CIDS3 workbook 
53.0% of the respondents take the same amount of time 
or less time to complete a case investigation record. 

Easier to Meet Program Standards, Most (77.4%) of 
the respondents agreed that "the workbook makes it 
easier to meet program standards" (table I-ll) . Only 
10 (11.9%) of the respondents reported that the work- 
book does not make it easier to meet program standards. 

TABLE I-ll 

Workbook's Effect on Meeting Program Standards 

Responses to the statement 
"The workbook makes it easier 



to meet 


program 


standards . " 


Response 
Category 


Respondents 
Number Percent 


Disagree Strongly 


1 


1.2% 


Disagree 


3 


3.6% 


Disagree Somewhat 


6 


7.1% 


Neither 


9 


10.7% 


Agree Somewhat 


21 


25.0% 


Agree 


36 


42.9% 


Agree Strongly 


8 


9.5% 


Total % Agree 




77.4% 



Number of Cases Contacts Stayed the Same . The last 
statement concerning effort is "Since using the 
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workbook I find that I make fewer contacts to complete 
a case.** Nearly half, 47.6%, of th'3 respondents 
disagreed (table 1-12) . 

TABLE 1-12 

Workbook's Effect on Number of Case Contacts 

Responses to the statement 
**Since using the workbook I find that I make fever 



contacts 


to complete 


a case. 


Response 


Respondents 


Category 


Number 


Percent 


Disagree Strongly 


8 


9.8% 


Disagree 


20 


24.4% 


Disagree Somewhat 


11 


13.4% 


Neither 


30 


36.6% 


Agree Somewhat 


6 


7.3% 


Agree 


6 


7.3% 


Agree Strongly 


1 


1.2% 


Total % Agree 




15.8% 



Only 15.8% agreed that they made fewer contacts. 
This indicates that staff are making as many or more 
contacts using the workbook as they did under their 
previous recording procedure. 

Summary — No Change in Overall Effort . In short, 
using the workbook took longer to document cases but 
made it easier to meet program standards. There was no 
cnange in the number of cases contacts. On balance, 
using the workbook produced no change in the level of 
effort required to investigate cases. 
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3t3t7 Open-Ended Comments 

and Suggestions for Improvements 



Comments (especially those in response to the ques- 
tion "how would you change the workbook?") provide some 
direction for resolving problems staff identified when 
they used the workbook. The most frequent response was 
a compliment or a statement similar to "no changes 
needed now." The following lists summarize respon- 
dents* recommendations for changes and their general 
reactions. 

Minor Changes . Respondents suggested the following 
minor changes: 

o Change the title Surmary of Referrals to Summary 
of Previous Referrels. 

o Add clerical tracking line(s) — (clerk's name and 
date of entry) . 

o Problems with continuity drew detailed sugges- 
tions: 

Summary of contacts, location good, but 
reading cases I found myself flipping 
back and forth to assessment pages* I 
would place description of injuries in 
section H where evaluation of children 
is, reverse G and H because injuries are 
described first in investigation. 



(Put) adult's explanation first as this 
is done (first). I think evaluations are 
made at a later point in investigations, 
so for continuity they should be later in 
format • 

o Add a space for complainant name and address. 

o Add a form letter to notify complainant of case 
finding. 

o For brief investigations, have an optional 
format with multiple people per page. 

o Add a place for school/alternate care addresses. 

o Add page numbers. 

o Leave largei margins to allow space for r' or 
top hole punch. 

o Improve definition of "extent" on risk-assess- 
ment page. 

o Give brief directions in workbook. 

Major Changes . Respondents suggested the following 
major changes: 

o Add a short form or system for recording simple 
and/or moved and/or invalid referrals. 

o Add more space and/or more pages to facilitate 
recording complex cases. 
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o Add a checklist of services to document services 
used and/or offered. 



General Reactions . Most of the praise for the 
workbook focused on how well it represented typical 
cases and the fact that the record is concise and easy 
to review. The most frequent negative observation was 
that the workbook had a fragmented appearance. Many 
staff noted that the workbook did not lend itself to 
recording investigations of complex cases. In par- 
ticular, they noted that institutional, day care, and 
sex abuse cases are difficult to record. 

A comment on the closing page of a Final Evaluation 
Survey does a good job of summarizing the comments 
about improving the workbook. 

I think the form is excellent for recording 
everything required by standards. I think I 
record so much more pertinent information that 
was omitted from the straight narrative, and 
that is good. However, it is very time- 
consuming and makes for longer recording times. 
I know the aim is to get to a computer system 
for recording, but so much of the information 
in these cases cannot be recorded with just 
check marks. I know we will continue to use 
the form in this area so we are all getting 
used to it. because our work load is getting 
heavier all the time we are not too pleased to 
have something that takes more time. You have 
done an excellent job with this, and — as with 
anything that is worthwhile — it takes time. 
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3r3>9 Summary— Workbook Evaluat ion: Reactions , 

Though Mixed. Tended toward Positive 

Staff reactionr; to the workbook are mixed but 
tended to be positive. Nearly 70 percent of respon- 
dents said the workbook vas a better method of documen- 
ting case investigations. Most pilot test staff agreed 
that the workbook is especially good for recording 
quickie and typical cases, but nearly half said that it 
was not good ^or recording complex cases. Comments 
indicated that the complex cases included investigation 
of sex abuse, day care, and institutional raferrals. 
Several staff suggested that the initial pages of the 
worJcbook be reorganised to reduce the need to flip 
among pages. They feel that the workbook can more 
closely reflect the sequence of an investigation. 
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3.4 EVALUATION OF TRAINING 



■Subsection 3,4 Directory- 



3 •4.1 Completing the Workbook and Understanding 
the Model (1-40) 

o One-third of trainees completed 
workbook inaccurately (1-40) 

o Accuracy lowest at first training 
s ite — Houston ( 1-41 ) 

o Understanding the model also lowest at 
Houston (1-42) 

o Summary: more and better training 
needed (1-42) 



3.4.2 Trainee Evaluations of the Workshop (1-43) 

o Instructors and material rated 
favorably (1-43) 

o Most and least helpful parts 
of training (1-44) 

o Suggestions for improving 
the workshop (1-45) 

o Summary: most rated the training 
favorably (1-45) 



3.4.3 Additional Training Needs (1-46) 



3.4.4 Relations between Training and Worlcbook 
Findings (1-47) 

o Staff with lower training scores over- 
estimated their proficiency in field 
use of the workbook (1-47) 

o Negative attitudes toward standardi- 
zation correlated with poorer under- 
standing of the RIF/RAF model (1-47) 



3.4.5 Summary: Evaluation of Training (1-51) 
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Four types of questions were used to evaluate the 
training (Appendix A). First, a test determined how 
well the trainees could complete a workbook exercise 
and how well they understood the RIF-RAF model. 
Second, the trainees were asked about the quality of 
the instructor's presentation and the training content. 
Third, the trainees were asked to identify the most and 
least helpful parts of the workshop. Finally, after 
training and after the pilot test they were asked open- 
ended questions to identify ways to improve the 
training. Subsections 3.4.1 to 3.4.4 present analyses 
of the answers to these questions, and subsection 3.4.5 
summarizes the findings. 



3.4.1 Completing the Workbook 
^nd Understanding the Model 

One-Third of Trainees Completed Workbook inaccu- 
rately . To test CPS specialists* ability to complete 
the workbook, they were asked to transcribe a paragraph 
of case narrative into a workbook. Workbook completion 
was scored on a scale from 1 to 3. Trainees who did 
not attempt or did not complete the exercise were rated 
0, and their scores were not considered in the 
analyses. If they attempted to complete the exercise 
but missed entries or recorded data incorrectly, they 
received a score of 1 — below expectation. A score of 
2, meets expectation, represents entry of a minimum of 
relevant data in the appropriate places and no incor- 
rect entries. If transcription into the workbook was 
exactly accurate the i^^ore was 3 — exceeds expectation. 

Table 1-13 shows the breakdown of scores. Six 
staff did not complete the exercise, and 27 ranked 
"below expectation." Thus, over one-third of the 
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trainees who completed the exercise were unable to 
satisfactorily complete the workbook. This finding 
indicates that training did not adequately prepare the 
CPS staff to complete the workbook accurately. 



TABLE 1-13 
Trainee Scores for Entering Information 
into the Workbook 



Exercise score 


Trainees 


Percent 


0 not completed 


6 


7.0% 


1 below expectation 


27 


31.4% 


2 meets expectation 


43 


50.0% 


3 exceeds ^isxpectation 


10 


11.6% 


Total 


86 


100.0% 



Accuracy Lowest at First Training Site — Houston . 
Calculating the scores across training sites shows that 
there are small differences in the ability of trainees 
at each site to transcribe information into the 
workbook • 

Trainees in Houston received training first and had 
the lowest average score (table 1-14) . After training 
in Houston, the curriculum was revised to include in- 

TABLE 1-14 
Trainee Scores by Location of Training 



Training Site Average Score Respondents 



Houston 


1. 


56 


25 


Rio Grande Valley 


1. 


77 


26 


Victoria 


1. 


76 


21 


Total 


1. 


69 


72 
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creased dialogue between the trainer and trainees and 
more emphasis on going through the workbook section-by- 
section* The difference in scores may be due to this 
increased emphasis on discussion and/or explanation of 
each section. Supervisors • scores (table 1-15) were 
approximately the same as the scores of CPS special- 
ists. 



TABLE 1-15 
Trainees Scores by Position 



Position 


Average Score 


Respondents 


Specialist 


1.68 


37 


Supervisor 


1*63 


8 


Understandina 


the Model Also Lowest 


at Houston 



Each of the three RIF factors (event, effect and dynam- 
ics) was scored separately. For each factor, the 
number of correctly marked cells was counted as "hits" 
and the number of incorrectly marked cells as "errors." 
Scores were computed for each factor as (2 * hits) - 
errors. The three scores were totaled to determine a 
score for understanding of the model. Scores are 
summarized in table 1-16. Scores on knowledge of the 
model are higher at Victoria and the Rio Grande Valley 
and lowest at Houston, where the first training session 
was carried out. 

Summary; More and Better Training Needed . 
Scores on the two exercises covering the content of the 
training were lower than expected. Trainees in Houston 
scored the lowest. Houston was trained first, and the 
training was revised based on feedback from Houston 
staff. The lower scores are probably due to the fact 
that the other sites received an inproved version of 
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TABLE 1-16 
Scores for Matching Model Sections 
and Workbook Sections 



Site 


Average Score 


Respondent 


Houston 


3.3 


25 


Rio Grande Valley 


7.5 


26 


Victoria 


5.8 


21 



the training. Average scores, for all trainees on both 
exercises indicate that more and/or better training is 
required to adequately prepare case investigation spe- 
cialists and supervisors* 

3,4.2 Trainee Evaluations of the Workshop 

Instructors and Material Rated Favorably * Most 
specialists and supervisors reported that the "in- 
structor demonstrated a genuine interest in this mate- 
rial." They also reported that the "instructor 
presented the material coherently,...." Table 1-17 is 
a breakdown of the actual responses. Four people 



TABLE 1-17 

Trainees' Opinions of Instructors and Material 



Response 


Interest 
in Material 


Coherent 
Material 


Definitely Yes 


30 


19 


Yes 


34 


33 


Neutral 


3 


11 


No 


0 


2 


Definitely No 


0 


2 
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reported that the training material was not coherent, 
and none reported that the instructor did not display 
an interest in the material. 

Most and Least Helpful Parts of Training . The 
training evaluation included two open-ended questions 
on the helpfulness of the workshop: (1) What part(s) 
of the workshop will be most helpful to you in doing 
your job? and (2) What part(s) of the workshop will be 
least helpful in doing your job? 

Forty-four CPS specialists responded to the first 
question . The description of the most helpful parts of 
the workshop fit into four categories: 

1. all of the training — 10 responses; 

2. step-by-step working through use of the work- 
book~9 responses; 

working on example (s) — 6 responses; 

4 . background and explanation of workbook and 
purpose — 5 responses. 

Ten of the responses did not address the question or 
could not be interpreted by the evaluator. 

Eighteen respondents identified the least helpful 
parts of the workshop . Nine of the responses fit into 
three categories used to identify the most useful 
aspects of the workshop; three other responses fit into 
a fourth category: 

1. all of the training — 3 responses; 

2 . step-by-step working through the workbook — 3 
responses; 
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3. working on examples — 3 responses; 

4. negative attitudes and comments of trainees — 3 
responses* 

Of the remaining 6 statements, 5 were praise for the 
workshop, and 1 individual reported that "presentation 
of the (RIF/RAF) model was dull and redundant." The 
statements concerning the examples were critical of 
their quality, not of examples per se, and two of the 
CPS specialists who criticized the examples identified 
examples as a way to improve the workshop (in response 
to another question) . 

Suggestions for Improving the Workshop « Six of the 
20 CPS specialist responding to the question "What 
suggestions do you have for improving the workshop" 
reported that they wouldn't know until they used the 
workbook. Four others used the space to note that the 
form needed to be linked with automation (not part of 
this pilot) . Nine responses were split between wanting 
more examples (5 respondents) and suggesting that the 
workshop curriculum needed to be better planned and 
organized (4 respondents) • One CPS specialist sug- 
gested a sound system and one la^^ge table for the 
trainees. 

Summary: Most Rated the Training Favorably > Re- 
sponses to the three open-ended questions tended to 
praise the quality and content of the training. This 
is consistent with the high ratings given for quality 
of presentation and material. In general, the trainees 
reported that the examples were useful and that there 
should be more of them . 
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3^4,? Additional Training Needs 



The final evaluation question "what additional 
information, support, or training would have been 
helpful?" elicited 35 comments from 86 staff (Appendix 
A) . Comments ranged from "if • • • using • • • computers 
(staff) would feel better about the system" to "how to 
use the workbook in real complex and difficult cases," 
Twenty-two of the comments requested three types of 
support (table 1-18) . 



TABLE 1-18 
Additional Information, Support, 
or Training Requested 



Type of Support Respondents* 

Clarification-Definitions-Instructions 10 

Follow-up Training and/or Meeting 7 

More Examples of Use 7 

Training was Adequate or Good 4 

Other 9 
*Two respondents each requested two types of support. 



Four staff stated that the training was adequate or 
good, and there were nine suggestions that were not 
supported by other comments or analyses. Of 86 pilot 
test staff who completed final evaluation question- 
naires, 31 reported that they would have liked more 
support. The general tone of the comments was that 
they had too much uncertainty concerning use of the 
workbook. 
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3.4,4 Relations between Training and Workbook Findings 



Scores from the workbook and the model exercise 
(described in subsection 3.4.1) were compated with 
several other data items collected during the evalua- 
tion. There are at least two useful findings among 
these relationships. 

Staff with Lower Training Scores Overestimated 
Their Proficiency in Field Use of Workbook . Pilot test 
participants were asked "About how many weeks did it 
take before you felt comfortable and prof i^cient in 
using the CIDSS workbook?" 

Staff who scored highest on the workbook exercise 
reported the highest average time to become proficient 
with the workbook after they began using it in the 
pilot test. Table 1-19 shows the average workbook 
exercise scores for respondents who reported that it 
took one to four weeks and more than four weeks to 
learn to use the workbook. Trainees whose scores 
reflected the most knowledge of how to use the workbook 
reported they took longer to learn to use the workbook 
in the field. Assuming that the training was useful 
and necessary, it is likely that staff who scored 
poorly and reported that they became proficient in a 
short time were using the workbook improperly. 

Negative Attitudes toward Standardization Corre- 
lated with Poorer Understanding of the RIF/RAF Model . 
Pilot test staff were asked on three occasions how the 
workbook would affect (or did affect) their jobs. 
Before training, staff were asked about their expecta- 
tions of a standardized guide. 
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TABLE 1-19 

Self-Assessed Proficiency vs. Tested Proficiency 
with the Workbook 



Time needed to become 
••proficient" in field 
use of workbook 



Average score 
on workbook 
exercise (during 
training) 



No. of 
Staff 



1 to 4 weeks 



1.4 



24 



5 weeks or more 



1.8 



20 



We are interested in learning how you think the 
use of a standardized investigation guide will 
change your job and the work you do. A number 
of possible changes are listed below. Please 
indicate your level of agreement or disagree- 
ment with each statement* by checking the 
appropriate numbered box on the seven-point 



1. Increase my workload 

2. Increase the freedom I have on my job 

3. Improve the quality of work I produce 

4. Make it more difficult to meet deadlines 

5. Make it difficult to do a good job 

6. Make my work more challenging 

7. Make my work more frustrating 

8. Decrease the discretion I exercise on my job 

9. Increase my ability to get work done 

10. Make it easier to keep up with my work load 

11. Make my job more interesting 



*For analysis, the 1 to 7 Likert scales were standard 
ized to make 7 the most desirable score, and the 11 
scores were averaged. 



scale. 
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After training, staff were asked to respond to the 
same statements terms of the workbook. 

Now that you are familiar with the CIDSS workbook, 
how do you think the workbook will change your job 
and the work you do? A number of possible changes 
are listed below. Please indicate your level of 
agreement or disagreement with each statement by 
checking the appropriate number on the seven-point 
scale. 

At the end of the pilot, staff were asked to rate 
the same 11 statements in terms of the effect of the 
wor)cbook on their jobs. 

We are interested in learning how the CIDSS 
investigation guide has changed your job and the 
work you do. A number of possible changes are 
listed below. Please indicate your level of 
agreement or disagreement with each statement by 
checking the appropriate numbered box on the seven- 
point scale. 

Table :c-20 shows that on all three occasions, 
train" ^,s who scored lowest on the model exercis were 
most likely to report that standardization and the 
workbook would or did have a negative impact on their 
job. 

The percentage of staff disagreeing, neutral, a. 
agreeing is fairly constant across the thiee evaluation 
surveys. There are at least two potential explanations 
for changes in the pattern of scores shown in table 
1-20. First, the low average exercise score of 3.7 is 
from staff expecting that standardization will have a 
negative impact on their jobs. The fact that the low 
scores tend to stay in the disagree row may indicate 
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that individuals with negative e;cpectations tended not 
to learn the RIF-RAF Model. 



TABLE 1-20 

Scores on Model Exercises versus Attitudes 
toward Standardization and the Workbook 
(Job Change Rating) 



Model Exercise Scores 



Job Change Before Pilot After Training After Pilot 
Rating^ score % resp score % resp score % resp 



disagree 
under 3.5 


3.7 


15.9% 


4.2 


14.3% 


4.8 


15.2% 


neutral 
3.5 - 4.5 


6.1 


43.2% 


5.1 


45.2% 


4.6 


50.0% 


agree 
over 3.5 


5.6 


40.9% 


5.1 


40.5% 


6.0 


34.8% 


number of staff 


44 




84 




46 



1. Averaged responses to the 11 statements listed on 
page (1-48). Higher scores indicate respondents think 
that standardization and the workbook will produce a 
positive change in their job sitations* 



Second, the highest average model exercise scores 
move from the neutral category, 6.1 before the pilot, 
to the agree category, 6.0 after the pilot. This 
finding indicates that understanding the model may 
contribute to a positive evaluation of the workbook. 
Taken together, these two findings support the need for 
training that explains the RIF/RAF Model so that staff 
have a good understanding of it. 

In short, individuals who had low expectations 
of standardization did not get a good understanding of 
the model. Staff who understood the model were likely 
to report, at the end of the pilot, that the workbook 
had a desirable influence on their job. 



1-50 

ERIC 88 



3,4,5 Summary; Evaluation of Training 



Staff who were trained in Edinberg showed the best 
understanding of the RIF-RAF Model and had the highest 
scores on the workbook exercise. Across the three 
sites, supervisors scored about the same as specialists 
on the workbook exercise and the RIF-RAF exercise. On 
one hand, the respondents gave the training a very 
positive evaluation. On the other hand, nearly a third 
of the pilot test staff reported that they would have 
liked more support in the form of definitions of terms, 
instructions, examples, and/or follow-up meetings or 
training sessions. The exercise scores support a 
finding that the training provided was not adequate. 
Analyses of training and workbook findings show that 
effective training is likely to facilitate acceptance 
and appropriate use of the workbook. 
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3,5 EVALUATION OF THE PILOT TEST 



Subsection 3.5 Directory 

3.5.1 Supervisors Rated Pilot Test Favorably 
on User and Setting Factors (1-53) 

3.5.2 Supervisors Also Rated Pilot Favorably 
on Innovation Factors (1-54) 

3.5.3 Pattern of Supervisors' Average Ratings: 
A Dip in the Middle of the Pilot Test 
with an Uptick at the End (1-57) 

3.5.4 Supervisors' Comments on the Implementation 
Factors Questionnaire (1-58) 

o Comments on innovation factors (1-58) 
o Comments on user and setting factors 
(1-60) 

3.5.5 Summary — Evaluation of the Pilot: Most 
Supervisors' Views Positive at End of 
Test (1-60) 
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The progress of the pilot test was evaluated by 
having supervisors complete the Implementation Factors 
Questionnaire (Appendix B) at the beginning, middle, 
and end of the pilot. The questionnaire tracks 24 
factors known to be important to successful implementa- 
tion of planned change. Supervisors read 24 statements 
that described the most desirable pilot test circum- 
stances. They were asked to note how accurately the 
statement described the CIDSS workbook pilot project. 
They indicated their level of agreement or disagreement 
with each statement on a. seven-point Likert scale 
ranging from strongly agree (1) to strongly disagree 
(7) with 4 as the neutral or don't know score. (Note 
therefore that lowey scores are favorable, higher 
scores unfavorable . ) 

The first 12 questions asked about user and setting 
factors that could influence the operation of the 
pilot. The next 12 questions, which asked about 
innovation factors , elicited supervisors' perceptions 
of the CIDSS workbook. The questionnaire had two pur- 
poses. First it allowed project management to identify 
and address problems and thereby to increase the like- 
lihood that the pilot would provide a good test of the 
workbook. Second, it identified problems and issues in 
implementation that could influence decisions on how 
and whether to implement the workbook in more sites. 

3.5,1 Supervisors Rated Pilot Test Favorably 
on User and Setting Factors 

For each user ind setting factor in table 1-21, the 
average ratings at the end of the pilot were better 
than at the beginning and midpoint. For two factors 
("3. It will be easy to retain experienced workbook 
users" and ''7. Current job descriptions cover required 
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roles") there was little change in scores over the 
course of the pilot. However, these scores are in the 
"agree" range, and these are two factors that project 
management had little or no opportunity to influence. 



3.5.2 Supervisors Also Rated Pilot Favorably 
on Innovation Factors 

Examination of supervisors' responses to questions 
on innovation factor? shows that their ratings of the 
pilot test improved over the course of its operation 
(table 1-22). Supervisors* ratings of innovation 
factors were best at the end of the pilot and worst at 
the midpoint. In all cases their average responses 
were between the neutral and agree scores. 
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TABLE 1-21 
Superv'isors • Assessments 
of User and Setting Factors 



User and Setting Factors Average Score* 

Week 1 Week 8 End 



1. Staff are aware of expected 

benefits, cost, & procedures. 3.0 3.0 2.5 

2. Staff have skills and 

knowledge needed • 2.5 2.5 2.0 

3. It will be easy to retain 

experienced workbook users. 3.4 3.4 3.1 

4. Staff perceive the need for 

using the workbook. 3.4 3.2 2.7 

5. Staff are motivated to give 

the worJcbook a fair trial. 3.3 3.0 2.5 

6. Staff accept the worlcbook as 

legitimate practice. 2.9 3.2 2.8 

7. Current job descriptions 

cover required roles . 2.4 2.5 2.3 

8. Facilities , equipment, and 

funds are available. 2.4 2.5 2.3 

9. Current procedures accommodate 

demands of the workbook . 2.9 3.4 2.3 

10. Leaders at all levels strongly 

endorse the workbook. 3.4 3.0 2.7 

11. Rules are in place to guide 

use of the workbook. 3.5 2.8 2.3 

12. I can identify and address 

factors that hinder proper use. 2.6 3.1 2.3 

Average 3.0 3.4 2.7 

Number of Sup rvisors 8 14 14 

*1 = strongly agree; 7 = strongly disagree; 4 is the 
neutral point or "don*t know** response. 
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TABLE 1-22 

Supervisors' Assessments of Innovation Factors 



Innovation Factors Average Scores* 

Week 1 Week 8 End 



1. Benefits, costs, procedures 

for use of workbook are clear , 3.4 3.3 2.5 

2. Procedures for getting, using, 

and storing are simple . 2*8 3.3 2.1 

3. The workbook configuration 

is stable . 3.6 3.9 3.1 

4. There is strong need for the • 

workbook at this site. 3.3 3.4 2.8 

5. Beyond meeting the iieed, the 

worJcbook has obvious advantages . 2. 5 3.0 2.3 

6. The effectiveness of the work- 
book is .observable . 3.3 3.0 2.3 

7. The workbook is fully developed 

and readily available . 3.6 3.7 2.9 

8. Workbook performance is 

highly reliable . 3.6 3.6 3.0 

9. The workbook is easy to 

maintain and upgrade . 3.0 3.4 2.5 

10. Accmisition costs for the 

workbook are quite reasonable. 3.3 3.4 2.9 

11. Operational costs for workbook 

use are low. 3.4 3.3 3.0 

12. Renewal costs are minimum. 3.3 3.4 2.7 



AVERAGE 3.0 3.4 2.7 

Number of Supervisors 8 14 14 

*0n the 7-point Likert scale, 1 = strongly agree; 7 = 
strongly disagree; (4) i^ neutral or don't know. 
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3fgf 3 Pattern of superv isors' Average Ratings; 

A Dip in the Middle o f the Pilot Test 
With an Uotick at the End 



The pattern of average ratings in tables 1-21 and 
1-22 was consistent across each of the three pilot 
sites: somewhat favorable at first, less so in Week 8, 
and most favorable at the end of the pilot test (figure 
1-3) . 



Most 

Favorable 1 — 

2 — 

3 — 

Neutral 4 — 

5 — 

6 — 

Least 

Favorable 7 — 

Week 1 Week 8 End of 

Pilot Test 

Figure 1-3. Pattern of Change in Supervisors' Views 
on the Workbook during the Pilot Project. The curve 
shows supervisors' average responses to both (1) user 
and setting factors and (2) innovations factors as 
gathered in the implementation factors survey. 

There are at least two potential explanations for 
the pattern. First, when supervisors completed the 
first questionnaire their staff had only one week of 
experience with the workbook. In other words, the 
supervisors may not havp received adequate feedback 
from investigators to fully critique the workbook or 
the pilot test procedures. As they gained experience 
they were better informed and better able to critique. 
As a result, the midpoint ratings were worse than the 
initial ratings. As the pilot progressed and problems 
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' 



2.7 



3.4 
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were solved, opinion became more positive, and the 
final ratings were better then in Week 1 and Week 8. 

Second, several substantive problems were identi- 
fied in Week 1 questionnaires. A memo responding to and 
resolving most of these problems was distributed at the 
same time as the Week 8 questionnaire. Responses to 
the Week 8 questionnaire identified fewer problems and 
showed proportionally fewer disagree ratings, but the 
average ratings were not as high as in Week 1. The 
poorer Week 8 ratings may be due to the fact that for 
the previous seven weeks the problems identified at 
Week 1 had not been resolved. Resolving them at Week 8 
could have helped cause the improvement" in the ratings 
at the end of the pilot test. 

3.5.4 Supervis ors' Comments on the 

Implementation Factors fIF) Questionnaire 

Supervisors were asked to explain any "disagree" 
response in a specific format. The request was "If you 
coded any responses in the darkened area please use 
this page to explain your response. In other words, if 
you do not agree with the statement please tell us (a) 
what has hapr *ed to cause you to disagree; (b) who or 
what organizational unit can resolve the problem; (c) 
what can be done to change future entries to agree." 

Comments on Innovation Factors . Explanatory 
comments on the workbook ( the innovation ) covered three 
general areas. 

o Definitions — Most supervisors felt that they 
needed more definitions for terms used in the 
workbook and clarification of just what informa- 
tion is appropriate to record in which sections 
of the workbook. Two supervisors pointed out 
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that they recognized that the pilot was intended 
to help define temns and clarify how the work- 
book should be used* 

o Changes — Several supervisors recommended changes 
in the workbook. Several noted that it tended 
to be bulky and repetitious. Substantive sug- 
gestions were to add a "moved" disposition; to 
add a collateral contacts page, and to develop a 
procedure for handling repeat referrals on an 
open case. 

o Quality of Casework — Supervisors expressed 
concern over the quality of casework done by 
investigators using the workbook. They reported 
a fear that the workbook focuses on standards, 
not on good casework. Most of these comments 
were punctuated with observations concerning 
good aspects of the workbook. 

The first two types of comments, need for defini- 
tions and for changes in the workbook, dominated the 
Week 1 IF Questionnaire comments but were absent from 
the IF Questionnaire given at the end of the pilot. 
Comments that raised the issue of quality of casework 
were the only type made in the end-of-pilot IF Ques- 
tionnaire and were absent from the Week 1 IF Question- 
naire. 

The absence of definition and change criticisms in 
the end-of-pilot IF Questionnaire may be due to the 
fact that as these substantive concerns were raised, 
the project director addressed and resolved them. 
However, during the pilot test it was not possible to 
address many of the concerns about quality. Also, as 
supervisors gained experience with the workbook they 
were collecting completed workbooks, which they could 
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compare with cases completed using other procedures. 
So, while there was not much basis for criticizing 
quality of effort at the outset of the pilot, by the 
end there was a set of workbooks to judge and compare. 
This may explain the fact that most of the "quality" 
criticisms were made at the end of the pilot. 

Comments on User and Setting Factors . Most of the 
explanatory comments on user and setting factors were 
made in the Week 1 questionnaires. There are several 
types of comments that can be summarized as follows: 

o Staff are motivated to use the workbook because 
they feel it is inevitable rather than thinking 
it is a good thing; some think it is bad. 

o The workbook is not cost-effective, and without 
computers it will not be. 

o Staff turnover is a problem. 

These comments represent a minority viewpoint, and 
they document the presence of some resistance to the 
workbook. It is notable that 8 supervisors wrote 11 
explanations at Week 1, and 14 supervisors wrote only 6 
comments at the end of the pilot. This observation is 
consistent with the finding that supervisors' ratings 
of the pilot test were highest at the end. 

3.5.5 Summary — Evaluation of the Pilot: 
Supervisors' Views Positive at End of Test 

The Implementation Factors Questionnaire docu- 
mented the progress of the pilot test and shows that 
(1) supervisors opinions of the workbook and the pilot 
test were positive and were most positive at the end of 
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the pilot; and (2) the workbook is at least a viable, 
and probably a good, alternative to previous methods of 
recording case investigations. 
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3,6 SUMMARY OF OVERALL FINDINGS 



Attitudes toward Work and Standardization . The 
pilot test of the workbook does not appear to have 
influenced staff attitudes toward work* Overall staff 
responses concerning the idc^^i of standardization and 
the workbook as the method of standardization remained 
fairly steady for the duration of the pilot • At the 
end of the pilot, 63% of the respondexits reported that 
they probably or definitely would use the workbook if 
the choice was theirs to make. 

Evaluation of the Workbook . Reactions to the 
workbook were mixed but generally positive. After four 
months of use 77.7% of respondents reported that the 
CIDSS workbook was a good idea. They reported that the 
workbook was as good or better than what they expected 
of a standardized guide. 

Nearly 70 percent of respondents said the workbook 
was a better method of documenting case investigations. 
Most pilot test staff agreed that the workbook is espe- 
cially good for recording c^uickie and typical case? 
but nearly half said that it was not good for recording 
complex cases. Comments indicated that the complex 
cases included investigation of sex abuse, day care, 
and institutional referrals. Several staff suggested 
that the initial pages (RIF) of the workbook be 
reorganized to reduce the need to flip among pages. 
They felt that the workbook can more closely reflect 
the sequence of an investigation. 

Most of the respondents (63.4%) agreed that they 
complete the workbook after the investigation. In com- 
parison with their previous procedures, 71.1% of the 
respondents reported that the CIDSS workbook took the 
same amount of time or less time to complete a case 
investigation record. Most (77.4%) of the respondents 
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agreed that "the workbook makes it easier to meet 
program standards." 

Evaluation of Training . Scores on the two exer- 
cises covering the content of the training were lower 
than expected. Trainees in Houston scored the lowest. 
Houston held training first, and the training was 
revised based on feedback from Houston staff. The 
lower scores are probably due to the fact that the 
other sites received an improved version of the 
training. 

Respondents gave the training a very positive 
evaluation. Trainees reported that the examples were 
helpful. But nearly a third of the pilot test staff 
reported that they would have liked more support in the 
form of definitions, instructions and examples, and/or 
follow-up meetings or training sessions. 

The workbooic and RIF-RAF exercise scores support a 
finding that the training provided was not adequate. 
Analyses of the exercise scores and workbook findings 
shows that the individuals who learned the most at 
training were most likely to have reported acceptance 
and appropriate use of the workbook. 

Evaluation of Pilot Test . The Implementation 
Factors Questionnaire documented the progress of the 
pilot test and shows (1) that supervisors' opinions of 
the workbook and the pilot test were positive and were 
most positive at the end of the pilot; and (2) that 
the workbook is a viable, and probably a good, alterna- 
tive to previous methods of recording case investiga- 
tions. 
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4. CONCLUSIONS 



The evaluation of the CIDSS manual workbook was 
designed to answer the following general questions: 

1. How do workers respond to use of the workbook? 

2. What are the problems with the layout or content 
of the workJDook? 

3. What types of training and support are necessary 
to support introd\iction and use of the workbook? 

The evaluation also set out to identify implications 
the worJcbooV" evaluation niight nave for tne content and 
organization of the CIDSS software. This section of 
the report discusces each of these questions and the 
associated f indings . 

How do workers respond to use of t he workboo k? The 
pilot test staff had generally positive reactions to 
the workbook. Nearly two-thirds of them said that tney 
would probably or definitely continue to use the work- 
book if it were their decision to make. 

Conclusion . Over three-quarters of the respondents 
agree that the workbook is a good idea. 

What are the problems with the layout or content of 
the workbook ? Responses concerning layout or content 
of the workbook were infrequent and did not identify 
any systematic problems. 

Conclusion . Several respondents identified small 
changes that did not influence the model or workbook 
organization. Project managers are evaluating the 
potential benefits of these changes. 
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What types of training and support are necessary to 
support introduction and use of the workbook ? A few 
problems were identified. Th<=i solutions to most of 
these problems consisted of better directions for using 
the workbook and more training, especially practice and 
examples. Frequently, training needs were specified as 
a "need for follow-up training." 

Conclusion . It is appropriate to conclude that 
given (1) detailed training, (2) written directions on 
use of the workbook, anc?. (3) training follow-up at one 
to two months, the workbook can and shoul oe imple- 
mented statewide. 

What are the implications for the CIDSS software? 
The format of the workbook was generally acceptable. 
Staff did not suggest revisions that influence the 
content or organizaicion of the software. Staff 
comments recommend a high level of training and support 
during workbook implementation. 

Conclusion . An implication of these comments is 
that the workbook and the software should be imple- 
mented separately to minimize the likelihood of train- 
ing overload. Results of the training evaluation are 
consistent with this conclusion. 
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EXECUTIVE SUMMARY 



The purpose of the Case Decision Project was to 
design a system to assist CPS staff in the investi- 
gation of child abuse referrals — specifically with data 
collection, decision making, and work load management. 
The project developed the Case Investigation Decision 
Support System (CIDSS), which consists of two parts: 
(1) the manual investigation workbook and (2) the 
automatec case investigation support system. Each 
component was designed to stand alone as an investiga- 
tion documentation system or, in concert with the 
other, to form a more comprehensive system. The manual 
system was evaluated in Part I of this report. Part II 
of this report contains the evaluation of the automated 
case investigation support system. 
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1, BACKGROUND 



I. I INTRODUCTION 

CIDSS' Place in CPS Aatomation « The Protective 
Services for Families and Children (PSFC) Branch has 
been engaged in an effort to automate Child Protective 
Services (CPS). This effort has been pursued in three 
stages . ^ 

o The first stage, providing automated support 
for the in*:,ake function, has been completed. 

o The second stage, automated support for case 
investigations, has been pilot -tested and is 
currently being redesigned for statewide imple- 
mentation* 

o The third stage, automated support for planning 
and managing casej that are opened for services 
is the most complex and sensitive of the three 
efforts and is still under development . 

The three systems are linked: each system pro- 
vides the initial input data for subsequent systems. At 
present. Automated MAPPER^ Intake provides the initial 
referral documentation required to .Initiate an investi- 
gation. The information is entered at intake, elec- 



A chart of CPS*s automation plan appeared in Part I, 
and others appear in the appendixes — e.g.. Appendix C. 
Variations among these charts refled. changes in the 
automation plan as development proceeded and more was 

earne'i about what the plan should entail. 

Maintaining, Preparing, and Producing Executive Re- 
ports — a computer language. 
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tronically transmitted to an investigation supervisor, 
and electronically assigned by the supervisor to a CPS 
specialist. The CPS specialist then reviews the intake 
information and begins using the recording features of 
the software. 

The software is backed up by the manual case 
investigation workbook, which is a paper representation 
of the software system. The software and the workbook 
are based on a model that increases the objectivity of 
an investigation. ITie workbook provides a standardized 
approach to investigation and a field recording medium 
and guide that reinforces the software structure. 

Purposes of CIDSS . The case investigation soft- 
ware and the workbook were developed to — 

1. Standardize recording of case investigations . 
The software provides a standardized format for 
recording and reporting investigation results. 
Standardization makes conmiunication easier and 
helps ensure complete and adequate case records. 

2. Support the collection (recording), analysis, 
and use of information pertinent to the decision 
to open or close a case . The software requires 
recording of pertinent data and prints or displays 
the information in formats that support appro- 
priate use of investigation findings and an objec- 
tive approach to decision making. 

3. Automate certain case management tasi..s . The 
system requires certain case management tasks to 
be performed by using the software: case assign- 
ment, case transfer, and case closure. These tasks 
can only be performed by the supervisor. The soft 
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ware automatically captures the date andtime that 
the task was performed and stores it asthe official 
time of completion. 



4« Provide automated management information 
reporting . The software gives on-line management 
information, which allows managers to track the 
status of intake and investigation cases and to 
generate management reports on demand. 

Several enhancements of this system have been planned 
or are being developed. Two major enhancements are — 

o electronic update of state office data bases; 
and 

o automatic recognition of risk profiles. 



1.2 CPG AUTOMATION PLAN 

For the past several years, human servic*^ agencies 
have found themselves in a situation of declining 
resources and expanding client service needs. One way 
to address this problem is to reduce the cost of ser- 
vice delivery and improve the effectiveness of services 
delivered. Computers have helped private industry 
reduce overhead and lower costs. DHS, like many other 
public agencies, has invested in automction in hopes of 
replicating private industry success. 

Information Needs . PSFC's automation goals in- 
clude development of an information system that meets a 
broad range of needs For the specialist in the field 
these needs include — 
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o a standardized format for recording; 

o quick retrieval, revision, and transfer of case 
information ; 

o automated production and transmission of re- 
quired forms; 

o support in meeting program requirements; 

o case decision support systems ; 

o work load and caseload status summaries; and 

For the supervisor, these needs include — 

o more easily understood and interpreted records; 

o automated systems to facilitate work load 
nanagement ; and 

o unit work load and caseload status summaries. 

For regional and state program staff these needs 
include- - 

o work load and performance summaries by program 
director, region, and branch; 

o ability to describe and track the client popu- 
lation; 

o management decision support; and 

o policy testing (modeling) capability. 



For several years, CPS automation has included 
statewide management information systems such as the 
Child Abuse and Neglect Report and Inquiry System 
(CANRIS) and the Social Services Management System 
(SSMS). CIDSS is an effort to provide field staff with 
direct access to automated support. 

Automation Plan Stages > As mentioned earlier, 
PSPC's automation plan proceeds in three stages. The 
following paragraphs give further details on the func- 
tions at each stage. 

The first stage, Automated MAPPER Intake (AMI), is 
software to automate and provide printing and tele- 
communications capability for CPS intake. Software to 
automate intake has been pilot tested and is currently 
in use by one intake site. As telecommunications and 
computer facilities become available, AMI will be 
implemented in additional sites. 

The second stage, the Case Investigation Decision 
Support System (CIDSS) has two complementary parts: (1) 
the CIDSS workbook and (2) the automated CIDSS. The 
workbook has been pilot tested and is the subject of 
Part I of this evaluation report. CIDSS software has 
been tested in the field. The automated CIDSS is 
currently undergoing design refinements. 

The third stage of soft.ware development includes 
automated support for planning and managing cases that 
are opened for services • These functions are cur- 
rently in development and will be available by the 
first quarter of 1988. 

Experience with the pilot test of software to 
automate intakes led the CIDSS staff to design the 
CIDSS pilot test as a cycle of prototyping and re- 
design. The prototyping method allows for the involve- 
ment of field staff in the design and implementation of 
the final system. Rather than write, pilot test, and 
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revise software, the project work group chose to de- 
velop a model of the investigation process, prototype 
it in paper form, then revise end retest it until a 
fairly stable format was developed. 

1.3 DEVELOPING CIDSS 

Subsections 1.3,1 through 1,3.4 describe develop- 
ment of CIDSS (see Subsection 1.3 Directory for subsec- 
tion titles and page numbers), 

Subsection 1.3 Directory 

1.3.1 Introduction to CIDSS Development (II-6) 

1.3.2 Development of Manual Workbook (II-7) 

1.3.3 Development of RIF/Rft^ Model (II -8) 

1.3.4 Software Design (11-10) 



1.3.1 Introduction to CIDSS Development 

Regional and state office staff determined that 
three essential tasks were required to meet the project 
objectives: 

o Task 1— specify the data elements that need to 
be collected during an investigation in order 
to arrive at a sound decision about case dispo- 
sition; 

o Task 2 — develop a model of the decision-making 
process; and 
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o Task 3 — incorporate results of tasks 1 and 2 
into an Instrument that allows for ease of data 
collection by the caseworker and ease of reading 
by the supervisor. 

1 .3*2 Development of Manual Workbook 

The first step in accomplishing these three tasks 
was to design a manual version of the Case Investiga- 
tion Decision Support System (CIDSS). This manual 
workbook was submitted for review and modification to a 
group of CPS experts from across the state. One 
skilled practitioner from each of the 10 DHS regions 
was chosen, and— after an extensive review of the 
clinical and research literature on CPS — the first 
version of the manual workbook was developed. 

After producing three revisions of the workbook, 
the work group agreed that no further development could 
take place without testing the workbook in the actual 
work environment. Field-testing would determine the 
final version of the data elements and the most useful 
format in which to display them. Field testing would 
also help the work group identify the factors necessary 
in making decisions during case investigation and as- 
sessment . 

A field-ready version of the workbook was pro- 
duced, and three sites in Texas volunteered to test it 
for 60 days. The workbook was introduced with only a 
basic overview, and the regional sites tested it for 30 
days. At that time their recommendations for modifica- 
tions were obtained and used to generate another revi- 
sion. The revised workbook was then introduced to the 
same sites; after a further 30 to 45 days of use the 
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regional sites' recommendations were again solicited, 
and an improved version of the workbook was generated- 



1.3.3 Development of RIF/RAF Model 

Once the workbook had assumed a usable and rela- 
tively stable form, the developers decided that it 
should be tested at another site with the additional 
element of providing thorough training- During the 
process of developing this training, the initial form 
of the decision model first emerged. Relying on the 
results of the literature review and an analysis of how 
the workbook had been used in field-testing, a model 
was developed and introdu'' ed as the focus of f.\e train- 
ing. 

The model, which later came to be called the 
RIF/RAF Model (Risk-Intensity Factors/Resource-Avail- 
ability Factors), was based upon the id3a that the 
decision to provide child protective services is a 
two-stage process (A chart of the model appeared in 
Part I of this report, and a slightly more detailed 
version appears in Appendix D), 

Stage 1 . Data from three general areas are col- 
lected and analyzed to determine the intensity of risk 
for abuse/neglect to the child. These areas are — 

o Event: did the alleged abuse/neglect occur? 

o Effect: how severe was the abuse/neglect, and 
what are its effects upon the child and the 
family? 
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o Environment: to what extent does the psycho- 
social and physical environment act to support 
or prevent the occurrence of abuse/neglect? 



Stage 2 , The second stage in the RIP/RAF Model is 
invoked only if some degree of risk intensity is deter- 
mined in Stage 1. Resources available to reduce risk 
intensity are assessed in order to arrive at one of two 
case decisions: (1) to close the case or (2) to open 
the case for in-home services or removal of the child • 
The case decisions are assessed as follows: 

o The Family: Does the nuclear and/or extended 
family have sufficient resources to reduce 
intensity of the risk? If so, the case can be 
closed. 

o The Community: Are community resources avail- 
able and accessible to the family to reduce the 
risk intensity? If so, the case can be closed 
after appropriate referrals are made. 

o .ild Protective Services: If the child is 
still at risk after the application of family 
and community resources, the family is eligible 
for child protective services. The level of 
risk at this point will determine the level of 
intervention (in-home services or removal). 

The workbook was introduced to the new site with 
training based on the decision model and was tested for 
60 days. Results from this test were presented with 
the results from the other field tests at a meeting of 
the work group. Another revision of the workbook re- 
sulted, and the work group determined that the workbook 
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was close enough to ics final form that a formal pilot 
test and evaluation was called for. The work group 
decided that the pilot tost should be conducted on all 
types of cases, in both rural and urban settings. Two 
regions that met the selection criteria voluntei?red as 
pilot sites: Region 11 (Houston) and Region 8 (Corpus 
Christi and the Rio Grande Valley). 

1,3>4 Software Design 

Software design began when the work group had 
specified the initial set of data items and produced a 
model workbook. The design proceeded through three 
stages — conceptual, general, and detailed design. At 
each stage decisions had to be about the hardware envi- 
ronment, software selection, and software functions. 

Conceptual Design , The conceptual design stage 
coincided with identification of the data elements and 
design of the workbook (Appendix E). The conceptual 
design identified desired system features and functions 
and began to narrow the field of hardware and software 
options available* 

General Design . The process of specifying the 
general design occurred while the workbook was being 
prototyped and pilot tested. Experience with the work- 
book had illustrated the value of prototyping and 
system flexibility. 

The importance of flexibility in the system was 
underscored by the need to revise the Prompted Intake 
System to make it useful to intake staff. Precursor to 
Automated MAPPER Intake, the Prompted Intake System 
software forced CPS specialists to follow rigid data 
entry pattern that did not always fit the language or 
sequence of information given by complainants. When 
revisions were needed to adjust the software to the 
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actual intake environment, the progranuning effort 
required was substantial. Recognition of this problem 
was one factor that led to reprogramming the intake 
software to be more flexible. 

Detailed Design . Detailed design was carried out 
when it was apparent that the workbook pilot findings 
would support the workbook content and organization. 
The software to be piloted wat? a set of screen images 
of the workbook sections, including data elements and 
text entry areas. Th'^ recording environment included 
prompting and help features. The system was designed 
to be easy to use and to improve the quality of case 
investigation recording and CPS management support. 

..4 PILOT SITE IMPLEMENTATION 



Subsections 1.4.1 through 1.4.3 describe intro- 
ducing the automated CIDSS into the pilot sites. 

^Subsection 1.4 Directory 

2.4.1 Implementation Strategy (11-11) 

2.4.2 Implementation Problems (11-12) 

2.4.3 Resolving Problems (11-13) 



1. 4.1 Implementation Strategy 

The implementation strategy for the pilot site had 
three stages. 

o Stage 1 . The pilot staff received an orienta- 
tion on the CIDSS manual workbook. The purpose 
was to familiarize them with the data elements 
and the RIF/RAF Model before they were intro- 
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duced to the automated system. Pilot site 
staff were trained on the CIDSS manual workbook 
and began using it for all investigations in 
March 1986. 

o Stage 2 > Intake staff were trained on tho 

Automated MAPPER Intake (AMI) System and began 
operating AMI before full implementation of 
CIDSS. This sequence had to be followed be- 
cause CIDSS cannot work unless AMI is function- 
ing satisfactorily. AMI began operation in 
June 1986. 

o Stage 3 . The plan was to operate CIDSS with 
only two investigation units for a trial period 
of 30 days. At that point, a decision would be 
made as to the advisability of expansion to 
other units. This plan ensured that any major 
problems would have a limited impact and could 
be corrected before wide-scale implementation. 

1,4.2 Implementation Problems 

Stages 1 and 2 were carried out satisfactorily, 
but px'Oblems were encountered in trying to limit CIDSS 
implementation to only two units. After a short time 
of using the AMI software, it was discovered that all 
units receiving intakes from AMI would have to use the 
automated CIDSS — 

o for caseworkers to get the new intake reports 
and 
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o for supervisors to track case assignments and 
status (by means of the management Information 
reports that AMI/CIDSS software can provide) • 

The discovery (that CIDSS would have to operate In 
all seven units) significantly disrupted the original 
Implementation plan. Shortage of hardware and Inexperi- 
ence with the software precluded full Implementation; 
Instead, the project had to improvise. 

1.4.3 Resolving Problems 

CPS state office staff resolved the probl'^m by 
devising a way for two units to lake full use of CIDSS 
while other units used CIDSS In a limited manner. The 
two units documented all investigation cases on CIDSS, 
thus creating the electronic management reports as a 
by-product of case documentation. The other units used 
CIDSS only to update certain information on the manage- 
ment reports. This procedure remained the rule for the 
rest of the implementation stage* 

In September 1986, CPS state office staff met with 
pilot staff to identify software problems and to 
specify changes needed. Although state office staff 
felt that they were not able to give CIDSS as thorough 
a test as desired, they felt they learned enough from 
the pilot to redesign the system to meet the pilot 
staff's needs. This arsessment and redesign stage is 
consistent with the prototyping methodology. (Appendix 
F lists the problems and specifications that were sub- 
mitted to programming staff. Modifications are being 
made on CIDSS to reflect the changes suggested.) 
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2. METHODS 



2>1 INTRODUCTION 

Section 2 discusses (1) the population and sample 
in terms of their adequacy to answer the research ques- 
tions and (2) the research design and research ques- 
tions. 



2.2 POPULATION AND SAMPLE 

Sample selection was constrained by the fact that 
in order for the automated CIDSS to function, an elec- 
tronic transfer of case data from the Automated MAPPER 
Intake (AMI) System was required. AMI was only in use 
in the Fort Worth and Arlington areas of DHS Region 5. 
As a result, the pilot test was carried out in these 
areas . 

The original implementation design was for CIDSS 
to be used in all seven of Region 5's CPS units. How- 
ever, problems encountered in the implementation pro- 
cess severely limited the number of staff who could use 
the system fully. (These problems were discussed in 
subsections 1.4.2 and 1.4.3). 

All seven CPS units did have some experienca with 
using the automated CIDSS, but there were three differ- 
ent levels of experience and involvement among these 
units. 

o The first level of involvement consisted of 
five units that used the aatomated CIDSS only 
for receiving new intakes from AMI. These 
units performed no documentation on the auto- 
mated CIDSS. 
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o The second level of involvement consisted of 
two workers in the sixth unit who documented 
their investigations using the manual CIDSS 
workbook. However, actual entry of the data 
into the automated CIDSS was performed by a 
data entry clerk. 

o The third level of involvement consisted of 
four workers in the seventh unit who were 
designated for full testing of the automated 
CIDSS software. 

The pilot sites represent mainly urban and sub- 
urban settings that are not necessarily representative 
of CPS offices in Texas. The stated purpose of the 
pilot was to refine and field test the software system, 
not to test its application statewide. Therefore, the 
project managers felt that the purpose of the pilot 
could be served without a representative sample of 
units or specialists. 



2.3 EVALUATION RESEARCH DESIGN 

The evaluation of the automated CIDSS pilot test 
was designed to answer the following general ques- 
tions — 

o How do workers respond to use of the automated 
CIDSS? 

o How doos the automated CIDSS influence CPS work 
load? 
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o What types of training and support are neces- 
sary to support introduction and use of the 
automated CIDSS? 

The original evaluation plan was based on the 
assumption that all CPS units would use the automated 
CIDSS fully and equally. Going by that assumption, the 
evaluation was carried out with all seven units • As 
discussed previously, the implementation plan ran into 
significant problems that had to be resolved. As a 
result, the implementation plan no longer proceeded as 
the evaluation plan had envisioned. 

Evaluation Strategy; Questionnaires * The evalua- 
tion was designed to operate as a management feedback 
system to assist in diagnosing and addressing implemen- 
tation problems • One strategy to investigate how 
effectively the system operated was to administer 
questionnaires and surveys to CPS case specialists and 
supervisors. A series of attitude surveys (Appendix G) 
asked staff about their experience with CIDSS, their 
reactions to standardization of case investigation 
record Veeping, and their opinion on aspects of CIDSS 
and their jobs. Another type of questionnaire, the 
Implementation Factors Survey (Appendix H), offered a 
simple procedure for anticipating implementation 
problems and monitoring their resolution. 

Evaluation Strategy — Individual and Group Discus- 
sions . A second strategy consisted of regularly sched- 
uled meetings to gain information for system evaluation 
and problem resolution. Informal contacts and group 
d'*scussions were organized around complaints and prob- 
lems • 
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Constraints . The first strategy (questionnaires) 
was constrained to a very small ^sample (two supervisors 
and four specialists). Not only was the sample size 
small I but it was impossible to determine which level 
of software each respondent had used. 
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3. FINDINGS 



3.1 INTRODUCTION 

The automated CIDSS consists of three features — 
o automated case reading, 

o automation of case management tasks, and 

o automated management information reporting. 

The evaluation exaunined (among other questions) which 
feature of the system was most valuable. 

During the first three months of the pilot test, 
field staff generally reacted negatively to the first 
two features but felt the third feature could be useful 
to them. When the pilot was evaluaced in September 
1986, state office and regional staff decided to focus 
available resources primarily on further testing and 
development of the management information aspect of 
CIDSS, which regional staff felt had proven to have 
greater value than the case documentation aspect of the 
system. 

3.2 CPS SPECIALIST AND SUPERVISOR C0^4MENTS 

The CPS specialists' reactions to CIDSS were — 

o Thete was not enough access to terminals when 
the specialists had time available to perform 
case documentation • 
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o There was a general feeling that more training 
and follow-up during the initial learning 
process would have helped the specialists use 
CIDSS more effectively* 

o CIDSS was too slow and cumbersome; other 

methods of case documentation, such as using a 
dictaphone, were more effective, 

CPS supervisor reactions to CIDSS were — 

o CIDSS is not an effective use of the special- 
ists' time. Their main function is face- 
to-face contact with clients, not data entry. 

o Supervisors agreed that additional training and 
follow-up support were necessary during the 
initial stages of learning. 

o Supervisors agreed that automated support for 
case assignment and case management was needed. 
However, they also felt that those features of 
CIDSS did not fully meet their needs end expec- 
tations in reality. 

3 . 3 TRAINING 

Evaluation of training indicated that it was 
in-sufficient to prepare CPS specialists and supervi- 
sors to use CIDSS fully. The majority of respondents 
reported that, while training material was adequate, 
additional material, such as a software user's guide, 
would have enhanced the learning process greatly. 
Field staff also felt that, in order to learn CIDSS 
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fully, thay should have been relieved of their existing 
work load so that they could concentrate on learning 
the system. Field staff also felt that trainers should 
be on-site for the first few weeks of operation to con- 
tinue individualized training sessions. 
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4. CONCLUSIONS 



The purpose of the Case Decision Project was to 
design a system that would assist CPS staff in the 
investigation of child abuse referrals • From the 
findings of the pilot site evaluation, the following 
conclusions can be drawn regarding the degree to which 
the following objectives were achieved. 

1 . Standardize recording of case investigations . 
CIDSS Project staff developed the CIDSS workbook 
as a standard method of recording case investi- 
gations* The workbook is described and evaluated 
(favorably, on balance) in Part I of this report. 

2. Su pport the collection ( recording )i analysis i 
and use of information pertinent to the decision 
to open or close a case * The primary need of 
field staff was to have an effective automated 
management information system (MIS). The first 
tier of automated systems development should be 
the installation of a MIS to meet basic informa- 
tion needs. The second tier is usually a decision 
support system, which analyzes and configures data 
from the management information system to support 
a range of management decisions. 

The CIDSS software was an attempt to provide a 
third tier of automation — decision support to 
worker staff, i.e., a system for collecting and 
analyzing detailed client circumstances to support 
decisions on individual cases. Based upon the 
results of the surveys, questionnaires, and com- 
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ments from state office and field staff* project 
staff concluded that the software was too advanced 
for a work environment that lacked experience with 
the first two •'tiers'* of automation • 

3. Automated certain case management tasks * Staff 
felx: that the automation of case management tasks 
was not flexible enough to prove useful to them* 
Tasks that CIDSS required the supervisor to per- 
form were in practice being performed or docu*- 
mented at times by clerical or worker staffs They 
also expressed a need for the system to allow the 
documentation of the task t take place at a later 
time than its performance* For example, a super- 
visor might need to assign a case to a worker 
verbally in an emergency, when use of the computer 
would be impractical. The flexibility needed by 
staff has been included in the redesign of CIDSS* 

4. Provide automated management information 
reporting * Field staff reacted much more posi- 
tively to this aspect of the software. They 
recognized the values of automated management in- 
formation in their cSay-to-day operations. The 
management support aspect of CILSS focused upon 
the limited range of operations within the scope 
of the project, and it was not initially intended 
to provide support for the full range of in- 
take/investigation management needs. Experience 
with the management information aspect of CIDSS 
led supervisors very quickly to recognize its 
potential benefits, despite the limitations of 
CIDSS in this regard. In their evaluation of 
CIDSS, supervisors expressed their need for man- 
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agement information, and CIDSS is now being re- 
designed to provide a broader range o£ management 
information on intake and investigation. 
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APPENDIX A 

Evaluation Surveys 
(CIDSS Workbook Evaluation) 
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CORTENTS 



Attitude Survey, Cover Memo A-2 

Attitude Survey A-2 

Evaluation Survey I, Cover Information A-10 

Evaluation Survey I A-12 

Final Evaluation Survey, Cover Memo A-36 

Final Evaluation Survey, Follow-Up Cover Memo A-38 

Final Evaluation Survey A-40 
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SUBJECT: 




TEXAS DEPARTMENT OF HUMAN SERVICES 
Attitude Survey: Pretest of Case Investigation Protocol 



TO: 



r 



FROM: 

1 1 John Theiss 1 
Research Design Specialist 

Jeffrey Anderson 
Research Specialist 
Organization Development Division 
State Office 503-E 



DATE: October 10, 1985 



As you may have heard, your unit is one of twelve that will participate ^n 
pretesting a new process of recording Child Welfare case investigations. 
This process is based on a Case Investigation Support System (CISS) 
workbook. This workbook will replace current case recording procedures for 
up to four months, November through February. 

We need your opinions and expectations concerning use of the workbook, 
vour responses to the enclosed survey will help us co evaluate the CISS 
workbook and our efforts to manage the pretest. The survey takes about 
ten minutes to complete. 

All responses to the survey will be kert confidential. To assure ttie 
confidentiality of your survey, do noL write your name on the survey form. 
Instead, please check that your name and mail code appear on the cover 
sheet. We need the cover sheet to be able to tell if each of the workers 
returned surveys. It will be removed from the survey when it is returned. 
In other words, John will recieve the survey. He will remove and throw out 
the coversheet with you name. Your name will not appear on the survey or 
with the responses stored in the computer. 



Please complete and return the survey as soon as possible. This survey 
an important source of infotaiation about your opinions and expectations 
concerning the CISS. Please contact us [(512) 450-3696 STS 887-3696 or 
3697) if you have any questions or if you would like a summary of the 
results when they become available next fall. 



IS 



Jeffrey M. Anderson 



John Theiss 



jMA: JTrnel 



Attachment 
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AN eCUAL OPPORTUNfTY EMPLOVER 



CASE INVESTIGATION SUPPORT SYSTEM 



ATTITUDE SURVEY 



Office of Research 



Demonstration & Evaluation 
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CASE INVESTIGATION SUPPORT SYSTEM 



ATTITUDE SURVEY 



GENERAL INSTRUCTIONS 

o Please answer e^ery question. If you like, you may discuss items in the comment 



o We want to know your honest opinions. Please do not put your name on the 
questionnaire • 

o Most of the questions ask that you circle one of several numbers that appear on a 
scale beside the item. You are to choose the one number that best matches the 
description of how you feel about the item. 

o Please read each question carefully. The scale descriptions are different in 
different parts of the questionnaire. 

o To return survey please fold in half with return address showing and staple 
closed • 



section. 



Thank you for your help. 




Please answer eacn of the questions below by circiinfe che nunber next to the descripcion 
wnicn besc fics you* 



1. Are you: 



I* Feaale 



2. Male 



Approxioately how aany years have you; 
woriceu wit a DriR? 

1. less Chan 1 year 

2* I to less Chan 2 

3* 2 CO less Chan 3 

4. 3 CO less Chan 5 

5. 5 CO less Chan 7 

6. 7 CO less Chan 9 

7. 9 or more years 



worKed wich CPS? 

1. less Chan L year 

2. L CO less Chan 2 

3. 2 CO less Chan 3 

4. 3 CO less Chan 5 

5. 5 CO less Chan 7 
7 CO less Chan 9 

7. 9 or niore years 



3. which of che following oesc describes you currenc posicioa? 
1. Worker 2. Supervisor 



00 you chink chac che use of i scancaraized iavescigacion guide is a GOOD IDEA for 
Chila Proceccive Services? 



CO 

z 

llJ 

CL 
X 

ill 



CO 

< 

C 

CL 
O 

O 



1 Definicely i«0 

2 Prooably NO 

3 Hot Sure 

4 Probably YES 

3 Definicely YES 



.;ere are some woras and phrases wnicn can be used co describe you presenc job. Circle 
cne numoer on each line chac describes how you see your job. For example, if you 
cainK you job is very "boring'* circle numoer L, righc nexc co che wora "boring." If 
• ou chink your job is very "incerescing," circle number 7, righc nexc co che word 
'Lncerescinti.** If you chink it is somewhere in becween, circle a number becween I 
and 7* 

iircie Che number chac besc describes your job. 
3e sure co circle a number on each line. 



ooring . . . . 
Enjoyable • . 
Useless • . . 
Friendly . . . 

Full 

Discouraging • 
Rewarding • • 
Brings Ouc The 
Besc In Me • 



2 3 4 5 6 7 Ir.terescin^, 

2 3 4 5 b 7 Miserable 

2 J 4 5 b 7 Worchwhile 

2 3 4 5 b 7 Lonely 

2 3 4 5 0 7 Empcy 

2 3 4 5 6 Hopeful 

2 3 4 5 b 7 Disappoincing 

Doesn'c Give Me 

2 3 4 5 6 7 .4 Chance 
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We are Interested in learniog how you think the use o£ a standardized investigation 
guide will change your job and the work you do. A number o£ possible changes are 
listea below. Please indicate your level o£ agreement o£ disagreement with each 
statement by checicLng the appropriate numbered box on the seven point scale. 



1* Increase my workload 

2. Increase the £reedom I have on my job . . . 

3. Improve the quality of work I produce . . . 

4. Hake it more di££icult to meet deadlines 

5. Make it difficult to do a good job • . • • 

6. Make my work more challenging 

7. Make my work more frustrating 

8. Decrease the discretion I exercise on my job 

9. Increase my ability to get work done . . . 

10. Hake it easier to keep up with aiy workload 

11. Make my job more interesting 



7. Are you familiar with the Case Investigation Workbook? 
[1] NO 
YES 

12 J Hy knowledge is minimal » but I am aware of the workbooks « 
[3 J I have heard abour. and/or discussed the workbooks. 
l4] I have seen drafts of the workbooks. 
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This Completes the questionnaire, we would appreciate any comments you would like to make, 
please write them in the space below. When you have finished, please fold in half with 
return address showing and staple dosed* 



COMMENTS: 



O 
O 
X 

> 

H 

cn 



m 
m 

X 

m 

CO 
R 
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Evaluation Survey I was designed to evaluate the pilot test training. It also 
collected opinion and attitude information after the pilot test staff had been 
introduced to the workbook and received training, but before they had field 
experience with it. This additional opinion and attitude information was 
collected for three reasons* 

First, trainee responses could be used to identify problems before the work- 
book was in use. The comparing post training results with baseline results 
can be used to show changes in opinions and attitudes from pre pilot to after 
training. Identifying changes in opinions and attitudes at this point was 
Intended to avoid complications that might result from reactions to the work- 
book content or organization. 

Second, if the training was evaluated as being less than adequate, the opinion 
and attitude responses (especially in contract to pre-training responses) 
could help identify training weaknesses and staff ne^.ds to properly implement 
the pilot* 

Third, collecting opinion and attitude information after training provided an 
opportunity to identify changes over the course of the pilot instead of only 
before and after it. Therefore, if there were problems, the evaluation had 
the potential for identifying whether the changes ' 2re due to the workbook and 
the model and/or their use. 

Evaluation Survey I was completed by trainees immediately after training at 
the three sites, Houston, Edinburg and Victoria. 
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C/aSe investigation support system 



CO 



Q. 

X 
UJ 



EVALUATION SURVEY I 



Office of Research 



UJ -:: , ^ 

< Demonstration & Evaluation 



c 

o 

o 



To insure Confidentiality, this page vill be torn off & discarded by ORDE, 

ERIC 



c^ame Mall Code 



CASE INVESTIGATION SUPPORT SYSTEM 
EVALUATION SURVEY I 



GENERAL INSTRUCTIONS 

o Please answer every question. If you like, you may discuss items in the 
comment section. 

o We want to know your honest opinions. Please do not put your name in 
the questionnaire. 

Thank you for your help. 
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tye MM «pe voy «xPr«tt«c »oii» opxiuens cenctrnxo^ your tiiP«ct«tions .cwui th« CISS ^tkoco^. Trw 
<oa;owxtt4; ^u«t:ioM wxi: t«:: m ikk th« ;rilnific «^Uct«c ^our «KtHrct«;ion* . Circie ;n« miao«r 
tti* attttMnt iMt b<t: o«tcrib«t your opiosent. 

I. Ta« CISS •orcbooK it: 

^ Mch p«tttr tiuQ I txptcttd 

: tOMvtuc b«Ct«r ttuA I trMCttd 

3 tkOut Wluit I •SfMttt4 

5 sMCli «ora€ tliM I txpccct^ 



Do you thiak that tbia Workbook ia a COOt IDU for Quid Frotoctivo Sonricoa? 



Dtfialtoly NO 
Protehlr NO 
Not Suro 
PrebAkly ttS 
OafxAitoij ICS 



UJ 

CO 

2: 

UJ 
OL 
X 
UJ 

Ui 



IMr tb.t 7^ w Uiliar with tho as$ Uorkhook. how do t«u thiok th. i^rkh-k will chaag. 
7.«r job aatf th. work do? A Mbor of pooalbl. chMgoa ara Uatod b^o«. fl^ iodicata 
7^ iM^i of v^rooMOt of dlMtrooMt with ooch atat-uit by chociaa, tho approfriAta 
«Bh«r m tha aovos poiat acalo. 



CO 



c 

lb 

SI 
o 
o 



1. lactoaao sy workiMd 

2. ZaicroAoa tho fr««doa X ho^ o« ay Job . . . 

3. lapro^ tho quality of work I produea . . . 

4. hiko It «oro difficult t« soot dooAllMo . 

5. haka it afficult to do « food job .... 

4. Homo ay Mork aora chaUoayiBf 

7. HU* ay work aora fr4«rrati&i 

5. IWcraaaa tho diacratioo I oxarciao 00 ay Job 
^- laeraaaa oy ability to got work aooa . . . 
IC. Maaa it aoaiar to naop up with ay ««orklo«d 
11. naaa ay job aora into r«a ting 
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::• mcirxc zoscuss 



*i MM potitiv« «Qc oa« 'M(«civ« «c«cc««nt or oescrvction a sou; :^c CISS >J««. in* 

vornhooft, 4M/or :ii€ wt*, i; is Mint is94.tMntto*, 



Ul 
CO 
2 
UJ 
Q. 
X 
UJ 

UJ 

< 

f—'te 2 RATAftsMA b«iow lacl tt ^a lAfocsaUoa froa «a ImtAibt am as laiilAl iair«aclcAClo« coqcacc. 
<0 

^flAAM raA4 CklA ImiorMClM Aa4 MtAr Ic la thm mpfrpptlMtm pJUc«« la chc •ctACft«4 wor&book. 
< 

C 

^^ Od Harco 7, 198) you wtrt Aaaiftiaa ch« folIowia( Priori:? : ^buaa iav«scicc:ioa. ftf S. 
^rrlvad ac school Mvtrtlr ^iata wicn blACk«n«« tyts and a blooAv mmr. The scftool nurse 4ilctcs 
0:*itc nr. HSimr Abuttd F«c«. Th« lactMa report concAintd ^r. jllAAr's work phone nuabitr am /ou 

caI1a4 hla i«aA4Uc«ly. 

Mr. ftlttr rtporctd ch«c h« w«a om/ cryiac to teach his sccp<-soa» Fcca. itm laporCAncc or 

soccaaa la ACA4«aic Aa4 spore* Accivtclea. He SAid chAC «xp<ccs nis son to <xcci m <v«rvchioc 

and puaiabaa hla a«vtr«lv waco h« do«a noc. K« rtpcactdlr capntsiztd :AAt nm dots not ?«c« ^nc 

Zhmrmiof dooa noc Abuao hla. 

Bafiaalag wlch cIm AlltfACloaA Aoccloa, pIaaaa aactr cbls lolcidl latrtaciKtcioo laforaacioo la Chc 
•ctacMd UarkbooA. 
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CASE INVESTIGATION 
SUPPORT SYSTEM 
WORKBOOK 





1 WORKER: v... IdATE ASSIGNED: 






Written 


Oral notification of law enforcement within 24 hrs? J-JES — NO 
report sent to law enforx-ement within 5 calendar days? ^^S —NO 
Supervisor contactf.d for approval within 24 hrs? iJ^ES —NO 
Investigation initiated within 24 hrs? OES — NO 




Oral' notification of law enforcement within 24 hrs? 
Written report sent to law enforcement within 5 calendar days? 

Investigation initiated within 10 calendar days? 


—YES —NO 
^IS —NO 
^IS NO 




Oral or written 


notification of law enforcement within 3 calendar days? 
Investigation initiated vrithin 10 calendar days? 


__yES —NO 
__yES NO 



I iPRinPTTv tTTI 
COMMENTS: 
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FAMILY CONSTEllATIOK 



KAME 




SDC 




S. Riser 


37 


M 


SF 


Riser 


28 


F 


MO 


Pfr*^ f^mith 


11 


M 


OV 











J5ai2E 


r 







































ALLEGATIONS 



TYPE CHUDREN 



DESCRIPnON OF ALLEGATICTiS 



OTHER SiannCRNT INFOPMRn^ 
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RECORD OF CONTACTS 



(Type of cxjntact) 



(Persons contacted) 



(Pertinent observations) 
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of (xrtact) 



RECORD OF CONTACTS 



(Persons contacted) 



) (Pertinent observations) 



EVftLUWION OF CHILDREN 



NAME & AGE 



CHILO(REN) SEEN BY WORKER? Y N 



IPREVIQLS ABtm^MFF^ 



Nooosd psyc^enot condition 
Diagnc^ed nentally retarded 
Diagnosed psycholog problem 
Ltd. intellectual ability 
Anxious/fearfu] 
Withdrawn/depressed 
Hostile/agcpcessive 
Suicide tendencies 
Other 



Nomal behavior 
Hyperactive 
Substance abuse 
Physically assaults others 
Sexual acting-out 
School problems 
Delinquent behavior 
Defiant/provoking b^iavior 
DistiadDe^unusxjal behaviorj_ 

Other 



Nomal develpEinent 
Below normal weighVheig^xt 
Delayed speedv^notor 
Delayed social developnent 

Other 



I PHYSICAL CrWDinCN/HIST^ 

Good pt^ical condition 
Prenature/low birth-weight 
Serious illness/injury 
Disability 
Poor hygiene 
Failure to thrive 
Malnutrition 
Skin rash/disorder 
Otter 



|PlVPrMrT>-rHTTJ^ PFTATTOl^SmP 



Normal interaction 
Bonding/attach, 'disruption 
RDle reversal 
Lack of nurtxare/stimulation 
Child^ afraid of parent 
Child unwanted 
C3iild scapegoated 
Qiild perceived negatively 

Other 
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No injuriei noted. 



DESCRIPTION OF INJURIES 



BONE 
BRAI 
BRUI 
BURN 
CONC 

DISL 
DISM 
□CPO 
HEMA 
HEMR 

INTL 
POIS 
SCAL 
SENS 
SEXL 



SKUL 
SPRA 
SUFF 
WELT 
WOUN 

OTHR 



INJURIES OF MULTIPLE AGES? 



Yes 



No 



PICTURES TAKEN? 



Yes 



No 



ALLEG 


AFF 


A$M 


EXPLANATION OF ALLEGATIONS 


ABAN 








ABUS 








EDUC 








EMOA 








EMON 








MEDI 








PHYS 








SEXL 








SUPE 
OTHR 









DEGREE OF AFFIRMATION 

1— Affirms abuse/neglect 

2— Partially affirms abuse/neglect 

3— Denies abuse/neglect 

4— No explanation 



ASSESSMENT OF EXPLANATION 

1— Explanation consistent with other facts 

2— Explanation possible, bjt unlikely 

3— Explanation inconsist. with other facts 
^--Unknown 
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EVT^LUATION OF ADULTS 



ACCESS TO CHILD 
Full-tiiTB 
Part-tiiTB 
Infrequent 
None 



IINDIVIDU - CHARACTERISTICS 
3 problems noted 
Psychologic ^1/atot. pacoblems 
Limited intellectual ability 
Lack of inpulse control 
Law self ^teem 
Suicide tendencies 
Svibetanoe abuse 
Problenns vdth the law 
History of physical assaxjlt 
Eistory of sexual assatilt 

Other 



jPARENTING FACTORS 
Good parenting skills 
Limited par^ting skills 
Unreal, expect, of children 
Inapprooriate discipline 
Other 



[RELAin 



Healthy/si^jportive relat 
Marital/paranour problens 
Sexual d^function 
• Other 



ISTRESS EaCPOIS 
Financial problesns 
Qiplpyment problans 
Health prdDleros/disability 
Recent divorce/separation 

Other 



ivicriMizATiON msrcsc£ 

No victimization history 
7ibused/r qrlected as child 
Sexual* , abused as dbild 



Abused 



Tpouse/paranour 
Other 

"CIAL ISOEaS T 
No isolati' n 
Sore isolation 
■vere isolation 



[ ' .TION TO WORKER 



Cooperative 
'Jncoc^5erative 
Lie/threatening 
Other 



j PAST ABUS 'GLECT OF CHILD 



ERIC 
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All allegations explained to parent/caretaker? — ^YES J_JlO 



ALL EG 



AFF 



ABAN 

ABUS 

EDUC 

EMOA 

EMON 

MEDI 

PHYS 

SEXL 

SUPE 

OTHR 



ASM 



F.X PLANATION OF ALLEGATIONS 



DEGREE OF AFFIRMATION 

1— Affinns aDuse/neglect 

2— Partially affirms abuse/neglect 

3— Denies abuse/neglect 

4— No explanation 



ASSESSMENT OF EXPLANATION 

1— Explanation consistent with other facts 

2— Explanation possible, but unlikely 

3— Explanation inconsist. with other facts 

4— Unknown 



fl<;<;F^^MrNT o p homf PNVTRONMENT 



Home visit made? 



YES 



NO. DATE: 



Home environment adequate to protect child(ren)? YES ^1 



NO 
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in] 
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m| 


m| 


inj 


m| 


m| 


m| 




m| 




ml 


inj 


ml 


m| 


m| 


inj 


^1 


^1 


^1 


^1 


^1 


^1 


^1 


^1 


^1 


^1 


^1 


^1 


^1 


^1 


^1 


^1 


^1 


^1 




^1 


^1 


^1 


^1 


^1 


^1 


^1 


^1 


rr>| 


rr>| 


rr>| 


rr>| 


ro| 


ro| 


rr>| 


roj 






ro| 








m} 


ml 


rr>| 








ml 


rr)| 


ro| 


rr>| 


rr>| 


rr>| 


rr>| 






CSJ| 














C\J| 


cvi| 


C\J) 


CSJj 


CSJ] 


C\J| 


CNJ| 




CVJ| 


CSJj 


CSJ| 


C\J) 


C\J| 


C\J| 




CSJj 


C\J| 


C\J| 
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ASSISSMENT OF RESOURCE AVAILABILITY 



FAMILY ABILITY TO PROTECT CHILD(REN) :. 

: is/are able to protect child(ren) on own. 

: will monitor situation to protect child(ren) 

: will work with CPS to- protect child(ren). 

: is/are unable to proteC child(ren), 

: see(s) no need to protect child(ren). 

: is/are unwilling to project child(r2n). 

: Other: 



COMMENTS: 



rflMMUNITY RESOURCES USED/NEEDED TO PROTECT CHILD(REN): 



CASE DECISION 



DHS ACTION NEEDED TO PROTECT CHILD(REN) : 

NONE: Close case OPEN: In-home services OPEN: Remove child(ren) 

WORKER COMMENTS: 



Date results of investigation explained to Parents/Caretakers:. 
Alleged victim(s): Complainant:, 



rfORKER SIGNATURE: 



DATE: 



SUPERVISOR REVIEW : 

Supervisor CONCURS DOES NOT CONCUR with worker's recommendation. 

SUPERVISOR COMMENTS: 



SUPERVISOR SIGNATURE: 



DATE: 



ERIC 
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SUmARY OF PREVIOUS REFERRALS 



REFERRAL NO. 


DATE OF REFERRAL: 


DAii cASi CLOSED. 


Ktr tKKAL 


1 T Pt 


rvTruT AC rAcrLinDv 
LaTlNT Ur CAdtWUKK 




ADAM 


urn T 


Closed at intake 




__A8US 


__PHYS 


In^honie services 




_EDUC 


__SEXL 


Child removal 




_EMOA 


__SUPE 


Family moved 




_EMON 


_OTHR 


_Other 


1 

1 



REFERRAL NO. 


DATE OF REFERRAL: 


DATE CASE CLOSED: 


REFERRAL TYPE 


EXTENT OF CASEWORK 


COMiENTS 


_ABAN 


_M£OI 


Closed at intake 




_ABUS 


__PHYS 


In-home services 




_EDUC 


__SEXL 


Child rcTOval 




_EMOA 


_SUPE 


Fanrily moved 




_EMOK 


_OTHR 


pother 













REFERRAL NO. 


DATE OF REFERRAL 


DATE CASE CLOSED 


REFERRAL TYPE 


EXTENT OF CASEWORK 


COWENTS 


_ABAN __fCDl 


^Closed at intake 




_ABUS _PHYS 


In-home services 




___EDUC _SEXL 


^Child removal 




_EMOA __SUPE 


Family moved 




___EMON _OTHR 


_Other 





! REFERRAL NO. 


DATF OF RFFFRRAl 


OAT? CA^r ri 05FD 


1 REFERRAL TYPE 


EXTENT OF CASEWORK 


COMMENTS 


_ABAN 


_MED1 


Closed at intake 




__ABUS 


_PHYS 


In-home services 




_EOUC 


_SEXL 


Child removal 




_EMOA 


_SUPE 


Family moved 




__EMON 


_OTHR 


__Other 






' REFERRAL 


NO. 


DATE OF REFERRAL 


DATE CASE CLOSED 


'■ REFERRAL TYPi^ 


rXTFNT OF CASEWORK 


COKMENTS 


. __ABAN 


_MEDI 


Closed at intake 




■ __ABUS 


__PHYS 


In-home services 




__EDUC 


_SEXL 


Child removal 




_EMOA 


_SUPE 


Family moved 




_£KON 


_OTHR 


Other 
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IV. HATCHING THE MODEL AND THE WORKBOOK 



The CISS system is based on Che investigation model presented this morning* The 
workbook was planned to correspond to the model. Several major sections of the 
model and the workbook are listed below* Please put an 'X' in every box that 
represents a match* 



Vorkbook Sections 



Hodcl 



EVENT: 



What happened 



EXTENT: 

How serious 



DYNAMICS: 

Environment 



Allegations 



Evaluation of Children 



Des cription of Injuries 



Exp laination of Allegation (adult) 



Ass essment of Resource Availability 



Coqu Bunity Resources Used /Needed 



Supervisor Review 




13 
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V. miKlNC FEEDiAOC SCaZOK 



I* Inscrjccor 4«aofittr«c«d « genuine lnc«r«ac In this •ftc«rlal« 



1 



Neutral 

3 



So 



Dtf Inittly 
No 
5 



2» iMtructor pr«»«nt«d the •actrial coh«r«otXy, •■phatlxlng Mjor point* and saklnt r«latlon«hlp« 
dear • 



Otflnltaly Ikflnlttly 
T«s Y«« Ntutral No No 

12 3^5 



3. What part(s) or tha vorkabop will b« aoac halpful to jou in doing your job^ 



U. What part<«) of tba workahop win ba laaat halpful in doing your job? 



5. What suggestions do you have for Inprovlng this workshop'' 



ERIC 
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This Completes the questionnaire, we would appreciate any comments you would 
like to make, please write them in the space below. When you have finished, 
please fold in half with return address showing and staple closed. 



COMMENTS : 
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TEXAS DEPARTMENT OF HUMAN SERVICES 



SUBJECT: Final Evaluation Survey: Pilot Test of Case Investigation Workbook 



TO: FROM: 

[~See Distribution List ~| ' John Thelss ~' 

Research Design Specialist 
Technical Resources Section 
Organization Development Division 
State Office, 503-E 



DATE: March 3, 1986 



The formal pretest of the CIDSS workbook ended February 28 th. Attached Is a 
Final Evaluation Survey. It contains questions concerning your current work 
situation and your opinions of the workbook. This survey, and earlier surveys, 
interviews and discussions will be used to evaluate the workbook and our 
training and coordination of the pilot test. A summary of the findings will be 
distributed to each of you by July. 

The survey takes about 20 minutes to complete. Please complete and return the 
survey as soon as possible. Please contact me ((512) 450-3697 STS 887-3697] if 
you have any questions. 

All responses to the survey will be kept confidential. To assure the 
confidentiality of your survey, do not write your name on the survey form. 
Instead, please check that your name and mall code are highlighted on the 
distribution list. We need the distribution list to be able to tell who 
returned surveys, fhe list will be removed from the survey when it is returned. 
In other words, I will remove and throw out the distribution list with your 
name. Your name will lot appear on the survey or with the responses stored in 
the computer. 




JTT:ctv 



Attachment 
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AN EQUAL OPPORTUNITY 6MPLOV6R 




TEXAS DEPARTMENT OF HUMAN SERVICES 



SUBJECT: Final Evaluation Survey: Pilot Test of Case Investigation Workbook 

• TO: _ FROM: _^ 

See Distribution List John Theiss 

Research Design Specialist 
Technical Resources Section 
Organization Developfment Division 
State Office, 503-E 

DATE: April 1, 1986 

On March I sent out a final evaluation survey* You have either: 
not returned your survey, 
or 

returned your survey without the- distribution list. 

If you have not conpleted and returned your survey, please complete and 
return the attached survey. 

If you completed and returned your survey but removed the distribution 
sheet please fold and return this distribution sheet. My address is on the 
reverse. 

Thank you, 




John Theiss 
JTTrctv 
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CASE INVESTIGATION SUPPORT SYSTEM 
FINAL EVALUATION 

Office of Research 
Demonstration & Evaluation 
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CASE INVESTIGATION SUPPORT SYSTEM 



FINAL EVALUATION 



GENERAL INSTRUCTIONS 

o Please answer every question. You may discuss items in the comment 
section. We are especially interested in your comments because 
they frequently tell us how to avoid or solve problems we had not 
anticipated • 

o We want to know your honest opinions. Please do not put your name 
on the questionnaire. 

o Most of the questions ask that you circle one of several numbers 
that appear on a scale beside the item. You are to choose the one 
number that best matches the description of how you feel about the 
item. 



o Please read each question carefully. The scale descriptions are 
different in different parts of the questionnaire. 

o To return survey please fold in half with return address showing 
and staple closed. 



Thank you for yojr help. 
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Pl««.« «nwer the que.tlon. below by circling the mmber of the de.crlptlon ^hlch best fits 
you. 



!• Did you attend workbook training at: 1. Harlingen 
Other: 



2. Victoria 



3. Houston 



2. Approximately how many years have you worked with CPS1 



1. less than 1 year 
2e 1 to less than 3 
3. 3 to less than 5 



4. 5 to less than 7 

5 . 7 to less than 9 
6* 9 or more years 



3. Which of the following best describes your current position? 
1. Worker 2. Supervisor 



What percent of your workload involves case investigations? 



1. 75Z, or more 

2. 50Z to 75Z 



3. 25Z to 50Z 
A. less than 25Z 



5. Here are so.e word, end phre.e, which can be used to describe PJ-"' f.^^Sl^^'you'' 
nu-ber on each line that describes how you see your job. Tor example. 

< « verv "borlne" circle number 1, right next to the word boring. If you think your 
foS is Ter^ -fntirLt^ig!" Circle noiberS. right next to the word "interesting." If you 
think it is somewhere in between, circle a number between 1 and 7 . 

Circle the number that best describes your job. 
Be sure to circle fi number on each line. 



Boring 1 2 3 ^ 5 6 

Enjoyabl 1 - 3 * 5 6 

Useless 1 2 3 ^ 5 6 

Friendly 1 2 3 A 5 6 

Full 1 2 3 A 5 6 

Discouraging ^ ^ ^ ^ ^ ^ 

Rewarding 1 2 ^ ^ 5 6 

Brings Out The . ^ , c a 

Best In M 1 2 ^ 4 5 6 



7 Interesting 

7 Miserable 

7 Worchvhile 

7 Lonely 

7 Empty 

7 Hopeful 

7 Disappointing 

Doesn't Give Me 
7 A Chance 
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6a. About how many weeks did It take for you to become familiar with the CIDSS workbook 
format? 

weeks 



b# About how many weeks did it take before you felt comfortable and proficient in using the 

CIDSS workbook? 

weeks 



7. Do you think that the use of a standardized investigation guide is a GOOD IDEA for Child 
Protective Services? 

1 Definitely NO 

2 Probably NO 

3 Not Sure 

A Probably YES 
5 Definitely YES 



8. Do you think that the CIDSS workbook you have been pilot testing is a GOOD IDEA for Child 
Protective Services? 

1 Definitely NO 

2 Probably NO 

3 Not Sure 

A Probably YES 
5 Definitely YES 



We are interested i!> learning how the CIDSS investigation 
the work you do* A number of possible changes are listed 
level of agreement of disagreement with each statement by 
numbered box on the seven point scale. 



guide has changed your job and 
below. Please indicate your 
checking the appropriate 



ERIC 



I. Increased my workload 

2* Increased the freedom I have on my job . • . 

3. Improved the quality of work I produce • • * 

4. Made it more difficult to meet deadlines . . 
5* Made it difficult to do a good job 

6. Made my work more challenging 

7. Made my work more frustrating 

8. Decreased the discretion I exercise on my job 

9. Increased my ability to get work done . . . 

10. Made it easier to keep up with my workload . 

II. Made my job more interesting 

12. Improved the documentation in case records . 

13. Increased time to complete documentation . . 

A-33 

















2 








































>* 


i 




























H 




§ 


Pi Cd 


s 






C/3 








a 




o 










S! 






b3 


b3 


U 




o 




H 


b3 


Cd 


Cd 




w 




CO 


oi 




oi 
















= a 


Cx3 




Cd 


^ 


< 


^ 


H 


Cs3 


l: 


Cd 








M en 




a: 










bi o 




c 






a 


a 




^ 


*s 


< 




2 


3 


4 


5 


6 






2 


3 


4 


5 


6 
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10. If it was entirely your choice would you continue to use the CTDSS workbook? 



1 Definitely NO 

2 Probably NO 

3 Not Sure 

A Probably YES 

5 Definitely YES 



11. We have heard many comments regarding pilot staff *s expectations of and experience in 
using the CIDSS workbook. We have summarized groups of these comments into general 
statements. Please circle the number that best represents your reaction to the following 
general statements. 



The workbook makes It easier to meet 
program standards. 

A workbook case record Is less clear 
than a record before the Pilot Test. 



o 

H 
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O 


o 
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6 


7 


5 


6 


7 



The workbook Is especially good for- 
Recording quickie cases 
Recording coBplaz cases 
Recording typical cases 



6 7 
6 7 
6 7 



The workbook record makes It hard to 
really understand the case. 

I refer to the wo.kbokk as 1 do the 
Investigation. 

I fill out the workbook after the 
Investigation Is substantially 
completed • 



12 3 4 5 6 7 



1 2 3 4 5 6 7 



1 2 3 4 5 6 7 



I write less when I use the workbook. 



1 2 3 4 5 6 7 



Since using the workbook I find that I 
make fewer contacts to complete a case. 
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Z* One or the most Important uses of a pilot test " opportunl >• for hindsights Please 

list below some things you feel that we could r.:. ..one better • 

We are especially Interested In: 

a. What additional information, support or tra' >.g would have been helpful? 



b. How would you change the workbook? Why? 
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Any other comments you have. ^ : you need more space. Whe 

finished, please fold in half ^ ' .1 address showing and stapl : 



146 
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COMMENTS 



Any oth«r conents you have. Add page If you need more apace. When you have 
flniahtd. please fold In half with return addreat shoving and staple closed. 



O 
O 

> 

H 

CO 
H 
J> 
H 
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m 

X 

m 

CO 

m 
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COMMENTS 



12* One of the «ost Ivportant uses of a pilot test Is the opportunity for hindsight. Please 
list belov to«e things you feel that ve could have done better* 



We are especially Interested in: 



a* What additional IrJomatlon, supporr or training would have been helpful? 

O 
O 



> 



0) 
H 
> 
H 

m 
m 

X 

m 
2 

m 



b. How would you change the workbook? Why? 
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If it v«8 entirely your choice would you cotttimie to use the CIOSS workbook? 

1 Definitely NO 

2 Probebly NO 

3 Not Sure 

4 Probebly TES 

5 Definitely TCS 



We have heerd uny coaaents regerding pilot steff's expectetiong of and experie^f in 
using the CIDSS workbook. We heve suMurlzed groups of these cosnents into genlil^sl 
ststSKsnts. Plesse circle 'he nuaber that best represents your re«cc:r« to thedoUovii 
general stateaents. 



The workbook aakes it easier to mee 
prograa standards. 

k ^forkbook case record is less clear 
than a record before the Pilot Test. 
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The workbook is especially good for: 
Recording quickie cases 
Recording coaplez cases 
Recording typical cases 



1 2 
1 2 
1 2 



The workbook record sakes it hard to 
really understaod the case. 

I refer to the workbokk as I do the 
investigation. 

I fill out the workbook after tne 
investigation is substantially 
coapleted. 

I write less when I use the workbook. 



1 2 3 4 5 6 7 



1 2 3 4 5 6 7 



1 2 3 4 5 6 7 



1 2 3 4 5 6 7 



Since using the workbook I find that I 
sake fewer contacts to complete a case. 



5 6 7 
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6«« Aboat hon May w^^a did It take for you to bccoiM fwilllT with th« CIDSS vorkbook 
fonMtt 



b. About how manr wc^a did It take before you felt comfortable and proficient In ualng the 
CIDSS workbook? " 

weeks 



o 



Do you thliic that tha uae of a atandardlzed Invaatlgaclon guide is a COOD IdSa for Child 
FrotectlTe SarTlceaT ^ 



I 
2 
3 
4 

5 



Daflnitely HO 
Frobably NO 
Not Sore 
frobably TES 
Definitely TES 



0) 

> 

m 
m 

X 

m 

Do you think that the CIDSS workbook you have been pilot testing is a GOOD SEA for Child 

FrotectlTe Services? J£ 

m 



Definitely NO 
Probably NO 
Not Sore 
frobably TES 
Def iQltely TES 



9. We are interested in learning hov the CIDSS investigation guide has changed your job and 
the work you do. A nuaber of possible chinges are listed belov. ?lease indicate your 
level of agreement of disagreeaent with each statcnent by checking tne appropriate 
miabered box on the seven point scale. ^ . 



OS 



< 

5 



< 
5 



1* Increased ay workload 

2. Increased the freedom I have on ay job . . • 

3. laproved the quality of work I produce • • • 

4. Maw^ it more difficult to aeet deadlines . . 

5. Made it difficult to do a good job 

6. Made ay work aore challenging 

7. Made ay work aore frustrating 

S« Decreased the discretion I exercise on oy job 
9« Increased ay ability to get work done ... 
10* Made it easier to keep up vith ay workload . 
11* Made ay Job aore interesting 

12. laproved the docuaentation in case records . 

13. Increased tiae to complete aocuaentation • • 



< 

t/i 

Q 
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2 
2 

2 
2 

2 
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< 



ur c 
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4 
4 
4 
4 
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4 
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6 
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6 
6 
6 
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FlMM .Mwcr th« qnaatlona b«loir by elrellog the mb«r of the deaerlptlon which beet flt« 



1. Did jrou «tt«nd workbook training at: 1. Harllngen 
Other: 



2. Victoria 



3. Houston 



2. Approximately how many years hrtve you voriced with CPS? 



!• less than 1 year 

2. 1 to leas than 3 

3. 3 to less than 5 



^» 5 to less than 7 

5. 7 to less than 9 

6. 9 or more years 



3. Which of the following best describes your current position? 
1- Worker 2. Supervisor 



4. What percent of your workload involves case investigations^ 



1* 75Xf or more 
2. 502 to 75Z 



3. 25: to 502 

4. less than Z5Z 



o 
o 

H 



X 

m 

2 

C/) 



5* 



Here ara some words and phrases which csu be used to describe you present job. Circle the 




Circle the nunber that best describes your job. 
Be sure co circle a nunber on each line. 

Coring 1 2 3 4 5 6 

"njoyable i 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 

Discouraging l 2 3 4 5 6 

Rewarding 1 2 3 4 5 6 

Brings Out Tne 

Bes: In Me i 2 3 4 5 6 



Interesting 

Miserable 

Worthwhile 

Lonely 

Emocy 

Hopeful 

. . . .Disappointing 

Doesn't Give Me 
7 A Chance 
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APPENDIX B 

Implementation Factors Inventory — 
CIDSS Workbook Evaluation 
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Description of Inventory B-2 

Flow Chart of Process of Planned Change B-4 

project Monitoring Implementation Factors Questionnaire • • • • B-6 
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Project Monitoring Implementation Factors 



The Innovation Acceptance Inventory is a simple procedure for anticipating 
implementation problems and monitoring problem resolution. It recognizes 
that there are two sources of implementation problems. Problaus can arise 
because of imperfections in the innovation — changes may be needed to make 
it more useful or easier to use. There are 12 items on the Innovation 
Acceptance Inventory which probe for information about possible problems 
with the innovation. Problems also caa arise because of the users — th^y 
may need additional support or encouragement to help them accept the 
innovation. There are 12 items on the Innovation Acceptance Inventory 
which probe for information about user acceptance. 

By completing the inventory in advance of actual project implementation and 
completing it periodically throughout the implementation phase, a project 
manager can anticipate problems and know when they have been resolved • The 
inventory is also a valuable tool for locating implementation lever- 
age — I.e. identifying features of the innovation or characteristics of the 
users that can be promoted to foster successful implementation. 

Attachea is a copy of the Innovation Acceptance Inventory (tailored for the 
current project) and a figure showing where it fits into the CIW portion of 
the Case Decision project. The figure shows that the inventory stands 
between actual implanenting actions on the part of workbook users and 
implementation support activities managed by the Project Director. As is 
suggested in the figure, the Innovation Acceptance Inventory can help to 
forge a strong link between goal identification and goal attainment « 

The Innovation Acceptance inventory will also be used to monitor the 
Implementation of the Automation Pilot* 

More detailed information concerning the design of and research basis for 
the Inventory may be found in the following journal articles or by 
telephoning Cynthia Roberts-Gray at (512)450-3749: 

Koberts-Gray, C. & Gray, T. (1983). Implementing innovations: A model to 
bridge the gap between diffusion and utilization. Knowledge: Creation, 
Diffusion, Utilization , 5, 213-232. 

Roberts-Grav, C. (.\9b5). Managing the implementation of innovations. 
Evaluation and Program Planning , 8, UO-00. 
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THE PROCESS OF PLANNED CHANGE 



GOAL 
ATTAINMENT 



Small group 
consiiltatlor 



IMPLEMENTING 
ACTIONS 












12 Innovation items 












INNOVATION 


and 




ACCEPTANCE 


12 User items 


/ 


y 


N 




jTraining & 
plan revision 



IMPLEMENTATION 
SUPPORT 
ACTIVITIES 



work group 




GOAL 
IDENTIFICATION 



/fuller case documentation 

^no decline in job satisfaction 

^increased capability for 
automation 

••better decision making by 
experts 

^decreased reliance on individual 
experience based expertise 

^worker satisfaction with workbook 

protocol completions 
use of completed protocols 



sefulness of the workbook 
user acceptance of the workbook 

daptations of the workbook 
facilitation of user acceptance 

statements of desired outcomes 

descriptions of steps that will be 
taken to effect the outcomes 
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PROJECT MONITORING 
IMPLEMENTATION FACTORS 

Office of Research 
Demonstration & Evaluation 
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Name 

(optional) 

Site of Training (circle one) 



Houston 

B-4 Edinberg 

Victoria 



GENERAL INSTRUCTIONS 



o Please answer every question. If you like, you may discuss Items In the 
comment section* 

o Most of the questions ask that you check one of several numbers that appear 
on a scale beside the Item* You are to choose the one number that best 
matches the description of how you feel about the Item* Items coded In the 
gray area require comments* Please use the comment format presented In the 
examples • 

o The questions refer to Information or Issues related to your organizational 
unit and codes should be marked only for your unit* Of course comments may 
be made concerning any Information and Issues that might affect the Pilot 
Test* 



THANK YOU FOR PARTICIPATING 
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I. INNOVATION FACTORS 



Kumrous faccors affect chc success of an innovacion. The twelve faccors liscsd 
below are ofcea liaicea vich successful projects • AT THIS POINT IK TIME, UOV 
ACCCRAIELY DO THESE STATDlEKTS DESCOBE THE CASE INVESTIGATION PROJECT?, 

Please iodlcace your level of agreeaenc or disagreenenc wich each scacenenc by 
circling Che appropriate numoar on c&e seven pome scale. 





























0 














u 














CO 






0 






















(0 
















< 




< 


ec 

< so 






a 








a 




0 


>i 






c 


v« ^ 


2 














3 














:o 








y 






a 


0 


u 


V. 


c 


— • k. 


UIU 


X 










w Z 








< 


in 


z z 


VI 







1. Expaccad benefits » cose, and 
procedures for using che 

workbook are clear [1] [2] [3] [U] (5] 16] [7] 

2. Procedures for gcccing^ 
conplacing, using, and scoring 

Che workbook are siaple [1] [2] [3] [A] (5] {6] [7] 

3. The workbook configurcion is 
scabla— I.e. ic's concenc, 
foraac, and procadures will 
noc hava co change auch over 

t.^^ [II' [2] [3] [4] .{5] (6J 17] 



^* Tliere is a scrong need for 

Che workbook ac chxs siCe [1] [2] [3] [4] 15] [6] [7} 

Beyond aaecins che need, che 
CO workbookhas obrloua advancases 

2 for chls uaic [1] [2] [3] [4] (5} [6] 17] 

LU 

ma effecCiTeness of chc 
^ workbooK is readily 

^ observable [1] [2] [4] (5J [6] [7] 

U 

All cottponencs of che workbook 
^ are fully developed and 

ceaoily available [1] [2] [3] [4] 15} {6] [7] 



CO 



Workbook parfonance is highly 



C 

o 
o 



relUble [1] [2] [3] [4] [51 (6] [7J 

The workbook is easy Co 

aaiacain and up-grad [1] [2] [3] [4J [5] (61 17] 



10. Acquisidon coses for the 

woricbooK are quice reasonable. • • [IJ [2] [3] [5] [6] [7] 

11. Qperacional coses' — ^.g* coses 
for paper, energy, & supplies 

for work book use are low [I] [2] [3] [4] [5] [6] [7] 

12. llencvA.1 coses* — e.g. replace* 
aenc 4 upgrade coses are 

ainiaal [IJ [2] [3] [4] {5] (6) [7] 
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If you coded Any responses in the darkened area please use this page to explain 
your response* In other words, if you do not agree with the statement please cell us 

A* what has happended to c^use you to 'disagree?' 

B« who or what organizational unit can resolve the problem. 

C« vhat can be done to change future entries to 'agree?' 

For example » two potential descriptions for a disagree code for question 2 are 

!• I disagree because workers are finding that the workbook is very 

difficult to use in sex abuse cases. I have been unable to help them. 
To resolve this problem the Project Director needs to reevaluate training 
and/or the usefulness of the workbook in sex abuse cases. 

2. I disagree because late delivery of blank workbooks has caused use of old 
forms. The regional office takes 2 to 3 weeks to meet requests for blank 
workbooks. Workers are having problems switching back to the workbook. 
Project Director needs to do refresher training. The branch could 
distribute workbook directly to the pilot units or unit supervisors 
could over-order and maintain large stocks of workbooks. 
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II. USFR AND SETTING FACTORS 



1. 



5. 



UJ 

a 

X 

111 

UJ 

< 



c 
u- 

O 

O 

10. 
11. 



12. 



The tucc€tt of an InnoTCClon it Linked Co char«ccerlscics of che seCClng ana Ctie 
users as well as characceriscics of che Innovacion* The tn^elve setting 
characceriscics listed below are ofcen linked vich successful projects. AT THIS 
POINT IN TIME, HOW ACCURATELY DO THESE STATDIENTS DESCRISE THE CASE 
INVESTIGATION PROJECT? 



Please indicate your level of agreement or disagreeaent with eacn scateoent 
checking the appropriate number on the seven. point scale. 



< 



eo 
c 

0 



Staff are sware of expected ^ 
benefits » cost, & procedures 
for using the workbook. [1 

Staff have skills k 

knowledge neeaea to complete 

k use the workbook [ 1 ] 

Ferfionnel turnover is low 

enough that it %fill be easy to 

retain experienced workbook 

uaers [1] 

Staff perceive the need for 

osin^ cne worKbook [1] 

Staff are highly motivated 

to give the workbook, a fair 

trial [1] 

Staff's beliefs & values aa&e 

it easy for them to accept the 

workbook as legitimate 

practice [1] 

Current job descriptions are 

adequate to cover use, 

supervision, k other required 

roles [1] 

Facilities , equipment, & 
funds are available to 
support use of the 

vorkbook [1] 

Current bookkeeping, 
personnel, budget, & resupply 
procedures accommodate demands 

of the vorkbook [1] 

Leaders at all levels strongly 
endorse the workbook [1] 

Rules are in place to guide 

use, security, & accounta* 

bility for che workbook [1] 

I can idencify and address 

faccors chat hinder proper use 

of the workbook in ay unit. ... [1] 

















a 








u 




o 








0 


0 






u 








M 








< 


< ? 














(3 








JZ 


y — 


3 








0) 




u 


u 


s 




B 




0 


4) Q 


0 


< 


CO 


2 Z 


CO 



[2] 

[2] 
(2) 

[2] 

[2] 

[2] 

[2] 

[2) 
[2] 



[3] 
[3] 

13] 



[4] [51 
[4] (51 



] [2] [3] [4] [51 16] 



[61 
[6] 



by 



so 

Cfl 



Gfl 

c 

o 



17J 



[3] [4] 15J [6J [71 



[71 
[7J 



[4] 131 [6] [71 



[2] 



13] [4] 15] [61 [71 



[3] [4] [51 (61 [7] 



[3] [4] 151 16] 17} 



[3] [4] [51 [6] [7] 



[3] [4] [5] [61 [71 



13] [4] [5] (61 (71 



[3] [4] (31 [61 (7J 
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If you coded any responses In the darkened area please use this page to explain 
your response. In other words, if you do not agree with the statement please tell us 



A. what has happended to cause you to 'disagree?' 

B. whc or what organisational unit can resolve the problem. 

C. what can be done to change future entries to 'agree?' 

For example, two potential descriptions for a disagree code for question 2 are 

U I disagree because the regional office takes 2 co 3 weeks co meet 

requests for blank workbooks. The branch could resolve che problazi by 
distributing workbooks directly to the pilot units. Also, anlt 
supervisors could resolve the problem by over-ordering and maintain large 
stocks of workbooks. . 

2. I disagree bee 'ise It requires a lot of unit time to foreward copies and 
original of various items to the Project Director. The Project Director 
coald resolve the problem by allowing us to copy materials needed at the 
unit and foreward all original forms to him. 
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APPENDIX C 
CPS Automated Systems 
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FLOW OF CPS AUTOMATED SYSTEM 



S 



cal Systems 



MIS 



C 
u 

H 
P 
L 
£ 
T 
E 
D 



F 
U 
T 
U 
R 
E 




No 



Worker 
Interven- 
tion 




No 



P 
R 
0 
J 
E 
C 
T 

(S) 





Case Investigation Workbook 

;a8e Investigation Softvar^ 
/^Decision Support System. 

& Topic Library 




Regional 
Data 
Sases 



Case 
Management 
and 
Planning 



No 




Foster Care ' 
Reporting U ' 
Eligibility 1 













— > 


of 
Care 
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APPENDIX D 
RIF/RAF Model 



164 



F4ATURgS OF THE SYSTgM, DECISIQM MOPCL 



.SYSTEM IS CONSTRUCTED TO REFLECT DECISION-MAKING PROCESS 



.WHAT IS THE CASE DECISION? 
.CLOSE 

.OPEN: IN-HOME 
.OPEN: REMOVE 



.CASE DECISION BASED UPON ASSESSME^^^ OF RISK irfTENSITY AND OF 
RESOURCE AVAILABILITY 



INITIAL ALLEGATIONS 



EVENT 



CHILD/PARENT EXPLANATION 



RISK 
INTENSITY 
FACTORS 



.EFFECT 



•INJURIES TO CHILD 



CHILD EVALUATION 



ENVIRONMENT 



ADULT EVALUATION 



HOME ENVIRONMENT 




DHS PROTECTIVE SERVICES 



COMMUNITY RESOURCES 



FAMILY RESOUCES 
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APPENDIX E 

Material Used in Developing 
the Conceptual Desirjn 
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r r or cn 

AUTOMATED STSTCNS 





Contlxmua 
i of. 
Care 
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PROJECT OBJECTIVES 



ORIGINAL OBJECTIVES 

.PROVIDE METHODS TO IMPROVE THE CONSIST^CY AND ACCURACY 
OF DECISIONS DETERniNING THE EXISTENCE OF ABUSE OR 
NEGLECT AND ELIGIBILITY FOR CHILD PROTECTIVE SERVICES; 

.PROVIDE METHODS TO IMPROVE THE ASSESSMENT OF NEED FOR 
CONTINUED SERVICES TO REMEDY PROBLEMS CONTRIBUTING TO 
CHILD ABUSE OR NEGLECT. 



T>4ESE OBJECTIVES WILL BE ACHIEVED THROUGH ; 

.FOCUSING THE INVESTIGATION ON THE IDENTIFICATION AND 
RECORDING OF INFORMATION THAT IS PERTINENT TO THE 
DECISION TO OPEN OR CLOSE A CASE FOR SERVICES; 

.PROVIDING T}C WORKER AND SUPERVlsOR WITH INFORMATION 
THAT WILL ASSIST THEM IN THE INTERPRETATION OF DATA 
COLLECTED DURING THE INVESTIGATION; 

.REDUCING REPETITIVE RECORDING OF INFORMATION IN 
MULTIPLE FORMATS. 



. PROJECT DELIVERABLES 

.PILOT OF AUTOMATE© SYSTEM 

.Standardizvd data collection 
.Investigation d«cision support 

.MANUAL SYSTEM FOR NON-AUTOMATED SITES 

.PROJECT EVALUATION 



. PROJECT DELIVERY DATE: FEBRUARY 28. 19SA 



4 

t 



flUTOnflTED 
DECISION SUPPORT 



IHFORMflTION 
SUPPORT 



m 
I 

U) 



CBSE 
IHVESTICflTIOM 
SUPPORT 
SYSTEM 



. riRNHGEllEMT 

SUPPORT 



\ 



\ 



\ 



riRNUHL 
WORKSHEET 



flUTOMflTED 
OflTft EMTPY 



MAJOR SYSTEM COMPONENTS 



ERIC 



169 



FEATURgS OF SYSTEMS MAJOR COMPONENTS 



■ STANDARDIZED DATA COLLECTION 

•MANUAL INVESTI5ATI0N WORKBOOK 
•Standalone capability 
•Automated input document 

•AUTOMATED DATA COLLECTION SCREENS 
•Same design as workbook 
•Optional entry by worker 

•CANRIS, CLIENT OUTCOME ASSESSMENT 



■ DECISION SUPPORT 

•DECISION SEQUENCE 

•Worker must complete 
•Information display 
•Factor weighting 

.RISK ASSESSiMENT 

.Automated recognition of risk profiles 
•Worker alerted to risk conditions 

.CASE DECISION SUPPORT 

.Mathefflatical decision model 

.Worker in-fcrned o-f hoM case compares to similar cases 



. MANAGEWENT SUPPORT 

.PROVIDES FOR TRACK I N6 OF INVESTIGATION PROCESS 
.MANAGEMENT REPORTING 
.SERVICE CONTROL COMPLIANCE 



. INFORMATION SUPPORT 

.GUIDE FOR INVESTIG'^TIHN ACTIONS AND DECISIONS 

."EXPERT" INFORMATtON 
. Policy 

.Sta^-f Development training material 
.Pro-fessional literature 



ERIC 



APPENDIX F 



Case Investigation Decision Support System 
Status Report on Automated CIDSS 
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QIBSLPILQLSTATUS.REPORI 



ECQfeig9S-iD*HSiD9-QIB§L§Qfi!!i!CS 



1. Currently the Case Directory serves as a useful tool for case identification 
and tracking the case status, but it does not contain all the infomation needed 
by the workers and Managers for Managing the investigation workload. 

The result of this is that there is little incentive for the unit supervisor 
to keep inforaation current on the Case Directory, and it tends to be perceived 
as an additional chore rather than as a Nanageiient aid. In its current forn the 
Case Directory does not replace the Manual case log that supervisors use for the 
investigation caseloady although it does replace the manual notification of the 
regional Hasterfile of case assignMent. 

Mith the addition of certain inforaation fields, the Case Directory will be 
able to replace More of the Manual case tracking systems currently in use, 
including the Monthly generation of worker, unit, progra». and regional 
statistics needed to Manage the prograM. This, in turn, will provide the 
incentive to staff to keep the inforMation current on the systen. 



2. The procedures involved in updating inforMation on the Case Directory are 
cuMbersoMe and inefficient. 

CIDSS was designed to keep the Case Directory infomation crrent as a by- 
product of the worker's docuMentation of the case, but at the pilot site only 
two of 10 units are using this approach. The other eight units are having to 
Manage the Case Directory by using the CIDS software in a Manner for which it 
was not designed, i.e., as priMarily a Hanagewent Information Systew. and this 
IS proving to be very unwieldy. 

Our -Nginal pilot iMpleMentation design called for Automated Mapper Intake 
(Afil) to begin first, followed by use of CIDSS by only two investigation units. 
The iMpact of CIDSS was to be assessed after 30 days, and a decision nade to 
expand its usage to other units or to Modify it before taking this step. 
However, soon after AHI was iMpleaented, it becaMe apparent that all units 
receiving cases froM the intake unit would have to use CIDSS in order to ensure 
ti^at cases initiated on AHI were received by the appropriate unit and acted cn 
in a tiaely Manner. 

This had a great iMpact on pilot site staff, since Many of thew had to begin 
using CIDSS without the fully adequate preparation. Because the case^ 
documentation process on CIDSS had as yet to be tested, and because of an 
insufficient nuMber of terMinals to support full iMplementation of CIDSS ^ith 
ten units, the decision was iade for CIDSS to be fully ihnpleMented in two 
units, with the other units using CIDSS only to Manage the the Ca^e Directory. 
This brought on another set of probleMS. 



on 

is 



In order to use CIDSS siiply »» an «IS, the supervisor Must first sign 
to CIDSS, assign the case to the norlier and then sign off. When the case 
conpleted, the worker iust sign on to CIDSS, update key information in the 
case, and sign off. The supervisor «ust then sign on to CIDSS again, update 
information on the case, and then sign off. This process is not conducive 
to efficient unit Management. 

Another difficulty inherent in using CIDSS in this manner is that certain 
functions which CIDSS restricts only to supervisors or to workers are often 
carried out and/or documented by -.lit secretaries. In fact it is the unit 
secretaries who are responsible for maintaining the current manual systems 
used for unit case management. Thus, in order to use CIDSS primarily as an 
niS requires that the unit secretary use the supervisor's and the worker s 
authorization to keep Case Directory information current. This is extremely 
time-consuming and raises issues about HAP PER security. 

The result of all this is that management of the Case Directory is 
perceived as more of a burden upon the unit than an aid, and no unit has 
been able to keep all Case Directory information updated. 



3. There is no audit trail for cases transferred from one unit to another. 

When a case is transferred from one unit to another, there is no way *or 
the receiving unit to know the origin of the case or when it was 
transferred. It appears in the receiving unit's Case Directory as a new 
casefbut it doec not necessarily appear at the bottom of the Directory, .s 
cases transferred from the Intake Unit do. This has caused confusion a^ong 
the units, and at times it has resulted in some cases not being recogn:.ed 
and acted upon in a timely manner. 



4. Entry of case information on the case by the worker or unit clerk has not 
been fully tested at this poinv. 

Some workers feel that it is too time-consuming for the« to enter their 
own cHe information, while others feel that it works sat.sfactor, y. The 
oJe u erk who is entering all cases into CIDSS fee s s e is ab e t 
enter the data as rapidly as she was able to do under the old syster. The 
point howeve?, is that there has not been enough experience w.t data entry 
5 C JSS to fo;m a conclusion about the value of this aspect o t e syste.. 
The pilot site staff them-lves do not want to disable this part of the 
pilot until more testing has been done. 



5. There continue to be problems with the terminals "locking up.' 

This seems to be related to the printing process, ftt the Ri'^^'-^'J^ 
offiJ; When this problem became acute it ^Heviated y tak-ng OP 
nrint tpri»inal off the DOPS and connecting it directly to the DLP. uiner 

sTouldt star^^^^^^^ «ith the software, lack of adequate user 

documentation, and/or inadequate problem resolution procedures. 
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6. Autottdtfd filing of CANRIS report is not yet ready for inpleMentation. 



Adding this capability to the systeii increase the ability of CIDSS to 
streanline the papermork aspects of the investigation. It will prevent the 
worker fro« having to fill out and call in the infomation on the 2202. It will 
also give us the opportunity to develop an efficient and effective way to 
autonate this function for the field staff. 



7. There is no current capability for adding subsequent intake reports to 
already open investigation cases. 

On sone cases, several referrals are received and sent to CIOSS on the sane 
case. This appears in the Directory as if there are several cases, when in fact 
there is only one case with several referrals. The supervisors need the ability 
to attach to an already existing case subsequent referrals which do not warrant 
a separate investigation. 
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Questionnaires for Evaluai^ion of 
CIDSS Automated System 
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Case Investigation 
Decision Support System 



FINAL FEDERAL EVALUATION 
CPS SPECIALIST QUESTIONNAIRE 



Office of Strategic Management, 
Research, and Evaluation 
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EVALUATION 



0? THE 



CASE INVESTIGATION DECISION SUPPORT SYSTE*' 



GENERAL INSTRUCTIONS 

PUdM r#«d tn# fol lowing lnttri»ct<on» carefully and tt>an co^lirht th# quest I onn a Ira. 



Plaa8# read tha qtiastlond carafuHy and ba aura to ana»ar a*ary question • If you 
HKa, you way discuss Itams In tha c oaiiant section* Add pages If necessary. 

We want io know your honest opinions- Please do not put your na«e on the question- 
naire. 

The survey Identification nutiber on this page helps us eonltor the questionnaire 
returns without placing your nmm on the questionnaire or In the computer flies. 

Most of th« questions ask that you mark one of several nuiibars that appear on a scale 
beside the I* em* You are to choose the one number that best mi»tches the description 
of how you feel about the Item. 

The scale descriptions are different In different parts of the questionnaire* So, be 
sure to read the special Instructions that appear with each saclion. 



T>4ANK YOU! 



SURVEY ID 
(1-4) 
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TWO COMPONEIITS MAKE UP TfC CASC OECJSiOW PROJCCT; 

1« Th« CM9m invest 1 9at loo Dr/clftlon Support Systom (CJDSS) 
Worw 3oK • 

2« Tho Auto««to<l C«M if>vostl9«tloi« D»clslo«) Support Syst^^ 
FIRST WE WAKT TO LEARN WHAT YGU THiNK AeOi/T THE CJOSS WORKBOOK. 



J« THE CiOSS WORKBOOK 



1« J havo coiipio^ 



;nvottl9«tront prior to this pliot tast). 



Invostlgatlofiis) using tho CJOSS workbook (IncJudlng 



2* About how mm\y nooks did It tsko for you to bocow>» fs^wlHar wllh tho CiOSS %rorkbook 

fOHMt? 

•ks or i am not famljiar with It 



3« About how Many wsoks did It tdk4 bofor^ rou im^^ comfortabJa and prof Iclont In using 
Tho C<OSS worktook? 

wooks 04* i aw not prof iclont In using It 



4« OvaraJ]» Oo yog t^lnk that tho use of tho CJOSS workbook Is a GOOO JOEA for Child 
Protoctlvo Sorvlcos? 

OEFiNJTELY NO ^OBABLY NO NOT SJIE M08A3LY YES OE^^JNiTELY YES 
fl) [2) (3) [4] [51 



5« Od you think that tho uso of a standardized Investigation guide . Is a GOOO .'OEA for 
ChMd Protective Services? 



JEFJNJTELY NO 
[ 1 1 



Pr OBABLY NO 
f 2 ) 



NOT SURE 
I 3 1 



PROBABLY YES 
[ 4 1 



DEFJNJTELY YES 
I 5 1 



6. is the CiOSS workbook better or worse than the recording system It replaced? 



MUCH WORSE 
( 1 1 



SOMEWHAT WORSE 
[ 2 ] 



ABOUT T>C SAME 
( 3 J 



SOMEWHAT BETTER 
( 4 I 



MUCH BETTER 
I 5 1 



7. if It was entirely your choice would you continue to use the CJOSS workbook? 



DEFJNJTELY NO PROBABLY NO NOT SURE PROBABLY YES 
[i: (2) 13) (4) 



oefjnjtely yes 

I 5 I 
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Jf you heve not used the Automated Systfn, $TOft> Skip to ttw comments on page 6. 



THE AlffOMATtO CASE INV£t>TIGATION DECISION SUPPORT SYSTEM (ClOSS) IS THE 
SECOND COMPONENT OT THE CASE DECISION PROJECT. IN THIS SECTION OF THE 
QUESTIONNAIRE WE WANT TO LEARN ABOUT YOUR EXPERIENCE WITH THE AUTOMATED 
CASE INVESTIGATION DECISION SUPPORT SYSTEM (Automated ClOSS). 



K I have coR^pJeted 



investl9atlon(s) using tha Automated CIDSS« 



2« About how Jong did It take for you to become fam' ^wlth the Automated CI OSS 
format? 

days or I am not fawl J lar with It. 



3« About how many weeks did It take before you felt comfortabJe and proficient In using 
the automated CI OSS? 

days or I am not proficient In using lt« 



A. OveraJJ, do you think that the use of the Automated CIDSS Is a GOOD IDEA for ChlJd 
Protective Services? 

DEFINITELY NO PROBABLY NO NOT SURE PROBABLY YES DEFINITELY YES 
11) (2) (3) 14) [5) 



5» Oo you think that the use of an Automated system to support Case Investigation Is a 
GOOD IDEA for Chlid Protective Services? 

DEFINITELY NO PROBABLY NO NOT SURE PROBABLY YES DEFINITELY YES 
M) (2) t3) (4) [ 5 ] 



6« Is the Automated CIDSS better or worse than. the recording system It repJaced? 

MXH WORSE SOMTWHAT WORSE ABOUT THE SAME SOMEWHAT BETTER MUCH BETTER 
[1) 12) (3) (4) [ 5 ] 



7. If Ir were entlrejy your choice , wouJd you continue ^ use the automated C'DSS? 

DEFINITELY NO PROBABLY NO NOT SURE PR06.SLY YES DEFINITELY YES 
M) (2) 13) (4) 15) 
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8* ^ 9r% \nfmrm%fm6 In Jearntng how use oi the Automefd CIDSS has changed your job and 
the «ork you do* A number of possible changes ere listed beJo«* 

PS^ase Indicate your level -of agreetnent or dlsagrxwe n t «tth each statement by marking the 
appropriate numbered box on the seven polnt-scale* 

NOTE: disagreeing with the stateineots neans that the rever»e of the stateiwent Is true* 



THE USE or CIDSS HAS: 

A* Increased the freedom i 

have on my Job 

3. Improved the quaJlty of 

work I produce 

C. Hade It more difficult to 
meet dead! Ines* •••••« 

0* Improved the documentation 
In case records 

E. Made It difficult to do a 
gcod Job 

F* Made my work more 

challenging 

6* Made my work more 
frustrating 

H* Decreased the discretion 1 
exercise on my Job. • • • • 

U Increased my ability to 

get work done • « 

J* Increased the amount of 
time needed to compJete 
documentation 

K« Hade It easier to keep up 
with my workload 

L« Hade my Job more 

Interesting 

M» Increased the amount of 
time avalJable for making 
ci lent/collateral 
contacts 
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9. t«9 have he«rd many cements regarding pilot staff *$ expectations of jnd experience \n using 
the autOMated CtOSSand the- workbook • ^ have summarized grc>uDS of these coments Into 
general statements* 

«>leasa mark the number that beet repr^^nts your reaction to the following general 
statements • 

MOTE: disagreeing with the statement means that the reverse of the statement !s true^ 
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program standards. 
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The automated CI OSS record focuses on 
















Information Important to the case decision 
















more than the previous method of case 
















recording* 
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A automated case record Is less clear 
















than a record before the Pilot Test. 
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In© aJTQfncTeo ^^lUdd rocorc gfvos a more 
















complete picture of a cas«> than the 
















previous method of case rea^dlng. 
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1 enter Investigation Information on the 
















automated CIDSS as 1 work the case. 
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1 use the CIDSS workbook *o keep a record 
















as 1 do the Investigation. 
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1 refer to the ClOSS workbook as 1 do 
















the Investigation. 
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1 ha*e a good understanding of the 
















Investigation model (RIF-RAF) presented In 
















ClOSS workbook Training. 
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to. If It w9rm entirely your choice, the comouter respoAded qulctily and was always 
avallaole^ would you continue to use the automated CI0SS7 



OEFtNtTELY NO 
I 1 ] 



PROBABLY NO 
[ 2 I 



NOT SURE 
f 3 ) 



PROBABLY YES 
[ 4 ] 



OeriNlTiLY YES 
[ 5 1 



11« If the computer responded quickly^ and was always avalleblet would the automated CiOSS 
make It easier to hieet program standards? 



DEFINITELY NO FROBko Y NO NOT SURE PROBABLY YES 
[11 (21 (31 (41 



DEFINITELY YES 
[ 5 1 



fit. TRAINING 



!• How effective was the training you received to prepare you to use the 
la. CIOSS WORKBOOK lb, AUTOMATED CIOSS 



K Worthless 

2m Inadequate 

3 « Adequate 

4. Good 

5* Excel lent 

9. Old net receive training 
(explain below) 



1. Worthless 

2m Inadequate 

3 • Adequate 

4. Good 

5* Excellent 

9. Old not receive training 
(explain below) 



2m If you rateo training as 3 or lower, what changes In the training would help to raise 
your rating to good or excellent? 
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COMMENTS 



of the most fmportant uses of a pilot test 1$ the oooortuofty tor hindsight. 
Please list below some things you feel could have done better, 

We are especially Interested In: 

U What additional Information, support or training would have been helpful? 



2* How would you change the automated CI0SS7 Why? 
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COMMENTS 



3* Any other commots you have* Add poges If you need more space* When you have 
finished^ please fold fn half wfth return address showing and staple closed* 
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John The Us 
Research Design SpeclaHst 
Technical Resources Section 
Research and Evaluation Division 
State Office, 252-E 
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Case Investigation 
Decision Support System 



FINAL FEDERAL EVALUATION 
CPS SUPERVISORS QUESTIONNAIRE 



Office of Strategic Management, 
Research, and Evaluation 



o 

ERIC 



G-ll 

m 



SUPERVISORS' EVALUATION 



OF THE 



CASE INVESTIGATION DECISION SUPPOf<T SYSTEM 



GENERAL INSTRUCTIONS 

Pleasa read the following Instructions carefully and then complete the q ,estlonnalre. 



Please read the questions carefully and be sure to answer «very question. If you 
like, you may discuss Items In the comment section. Add pages If necessary. 

We want to know your honest opinions. Please do not put your name on the question^ 
naire* 

The survey Identification number on this page helps us monitor the questionnaire 
returns without placing your name on the questionnaire or In the computer files. 

Most of the questions ask that you mark one of several numbers that appear on a scale 
beside the Item. You are to choose the one number that best matches the description 
of how you feel about the Item. 

The scale descriptions are different In different parts of the questionnaire* So, be 
sure to read the special Instructions that appear with each section- 



THANK YOU I 



SURVEY (D 
(1-4) 
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TWO COMPONENTS WKE UP THE CASE DECISION PROJECT: 

1* The Case Investigation Decision Support System (CIDSS) 
Workbook* 

2. The Automated Case Investigation Decision Support System 
FIRST WE WANT TO LEARN WHAT YOU THINK ABOUT THE CIDSS WORKBOOK. 



\. THE CIDSS WORKBOOK 

1» I have reviewed and signed off on InvestIg«jtIon(s) completed using the 

CIDSS »K)rkbook (Including Investigations prior to this pilot test). 

2. About how many weeks did It take for you to become fami I lar with the CIDSS workbook 
format? 

weeks or I am not familiar with It 



3. About how many weeks did It take before you felt comfortable ano proficient In using 
the ClOSS workbook? 

weeks or I am not proficient In using It 



4. Overall, Do you think that the use of the CIDSS workbook Is a GOOD IDEA for Child 
Protective Services? 

DEFINITELY NO PROBABLY NO NOT SURE PROBABLY YES DEFINITELY YES 
[1] [2] [3] [4] [3] 



5. Do you think that the use of a standardized Investigation guide. Is a GOOD IDEA for 
Child Protective Services? 

DEFINITELY NO PROBABLY NO NOT SURE PROBABLY YES DEFINITELY YES 
[11 (21 t3] [4] (31 



6. Is the CIDSS workbook better or worse than the recording system It replaced? 

MUCH WORSE SOMEWHAT WORSE ABOUT THE SAME SOMEWHAT BETTER MUCH BETTER 
(1] (21 (51 (41 (3] 



7. If It was entirely your choice would you continue to use the CIDSS workbook? 

DEFINITELY NO PROBABLY NO NOT SURE PROBABLY YES DEFINITELY YES 
(11 (21 (31 (41 (31 
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lU AUTOMATED CIDSS 

If you have not used iUe Automated System, STQP> Skip to page 7 question 2. 



THE AUTOMATED CASE INVESTIGATION DECISION SUPPORT SYSTEM (CIDSS) IS THE 
SECOND COMPONENT OF THE CASE DECISION PROJECT. IN THIS SECTION OF THE 
QUESTIONNAIRE WE WANT TO LEARN ABOUT YOUR EXPERIENCE WITH THE AUTOMATED 
CASE INVESTIGATION DECISION SUPPORT SYSTEM (Automateci CIDSS). 



1. I have revlewd and signed off on Investlgatlon(s) completed using the 

Automated CIDSS. 



2. About how long did It take for you to become fami I lar with the Automated CIDSS 
format? 

days or I am not familiar with It. 



3. About how many weeks did It take before you felt comfortable and proficient In using 
the automated CIDSS? 

^ days or I am not proficient In using It. 



4. Overall, do you think that the use of tlte Autom.'^ted CIDSS Is a GOOD IDEA for Child 
Protective Services? 

DEFINITELY NO PROBABLY NO NOT SURE PROBABLY YES DEFINITELY YES 
(11 121 (31 (41 (51 



5. Do you think that the use of an Automated system to support Case Investigation Is a 
GOOD IDEA for Child Protective Services? 

DEFINITELY NO PROBABLY NO NOT SURE PROBABLY YES DEFINITELY YES 
(11 (21 (31 (41 (51 



6. Is the Automated CIDSS better or worse than the recording system It replaced? 

MUCH WORCE SOMEWHAT WORSE ABOUT THE SAME SOMEWHAT BETTER MUCH BETTER 
(11 (21 (31 (41 (51 



7. If It were entirely your -holce, would you continue to use the automated CIDSS? 

DEFINITELY NO PROBABLY NO NOT SURE PROBABLY YES DEFINITELY YES 
(11 (21 (31 (41 (51 



ERLC 



8« We are Interested In learning how use of the Automated CIDSS has changed your Job and 
the work you do* A number of possible changes are lls^ed belowi 

Please Indicate your level of agreement or disagreement with each statement by marking the 
appropriate numbered box on the seven polnt-scale« 

NOTE: disagreeing with the statements means that the reverse of the statement Is true* 



THE USE OF CIDSS HAS: 

A* Increased t!ie freedom I 

have on my Job* 

B. Improved the quality of 
work I produce « 

C. Made It more difficult to 
meet dead! Ines 

D* Improved the documentation 
In case records 

E. Made It difficult to do a 
good Job 

F« Made my work more 

chal lenging 

G« Made my work more 

frustrating •••••••• 

H* Oecreaseo the discretion I 
exercise on my Job 

I* Increased my ability to 

get work done 

j« Increased the amount of 
time needed to complete 
documentation • « 

K. Maoe It easier to keep up 
with my workload* « • • • i 

L* Made my Job more 

Interesting 

M« Increased the amount of 
time available for making 
cl lent/col lateral 
contacts * 
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9* We have heard many comrants regarding pilot staff's expectations of and experience In using 
the automated CIDSSanr, the workbook. We have summarlied groups of these comments Into 
general statements* 

Please mark the number that best represents your reaction to the following general 
statements. 

NOTE; disagreeing with the statement means that the reverse of the statement Is true. 



The automated CIDSS makes It easier to meet 
program standards. 

The automated CIDSS record focuses on 
Information Important to the case decision 
more than the previous method of case 
recording. 

A automated case record Is less clear 
than a record before the Pilot Test. 

The automated CIDSS record gives a more 
complete picture of a case than the 
previous method of case recording. 

The automated ClOSS Is especially good for: 

Recording quickie cases 

Recording complex cases 

Recording typical cases 

The auromated CIDSS record makes It hard 
to really understand the case. 

1 enter Investigation Information on the 
automated CIDSS as I work the case. 

I use the CIDSS workbook to keep a record 
as I do the Investigation. 

I refer to the CIDSS workbook as 1 do 
the Investigation. 

I have a good understanding of the 
Investigation model (RIF-RAF) presented In 
CIDSS Workbook Training. 
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}0. If It wre entirely your choice, the computer responded quickly and was always 
available, would you continue to use the automated CI0SS7 

OEFlNliaY NO PROBABLY NO NOT SWE PROBABLY Yf-S OeFINITaY YES 
11] 12) (3) [4) [5) 



11* If tt)e computer responded quickly, and was always available, would the automated CIDSS 
make It easier to meet program standards? 

DEFINITELY NO Pr^OBABLY NO NOT SURE PROBABLY YES DEFINITELY YES 
[ 1 ) 12) 13) [41 [ ^ ] 



IH. TRAINING 

}. How effective was the training you 
la. CIDSS WORKBOOK 

1* Worthless 

2* Inadequate 

3* Adequate 

4* Good 

Excellent 

Did not r«:elve training 
(explain below) 



Ived to prepare you to use the 

lb. AUTOMATED CIDSS 

1. Worthless 

2* Inadequate 

3* Adequate 

4« Good 

Excel lent 

9. Did not receive training 
(explain below) 



2» If you rated training as 3 or lower, what changes In the training would help io raise 
your rating to good or excel lent? 
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WE ARE CONCERNED ABOUT HOW THE AUTOMATED SYSTEM AFFECTS HOW YOU DO YOUR 
JOB. WE ARE PARTICULARLY INTERESTED IN THE FaLOWiNG THREE AREAS. PLEASE 
MARK A CODED RESPONSE TO EACH OF THE QUESTIONS AND PLACE YOUR WRITTEN 
DESCRIPTION IN THE SPACE PROVIDED, AND ON ADDITIONAL PAGES IF YOU NEED 
THEM. 



IV. SUPERVISOR QUESTIONS 



1. The automated CIDSS has case assignment and tracking capability built Into It. Please mark 
your level of agreement or disagreement with the following statements concerning the CIDSs 
managmenet features of casw a:»^?gnment and tracking. NOTE: disagreeing with the statement 
means that the reverse of the statement Is true. 



The automwited CIDSS makes ft easier for me 
to track CPS specialists worMoad. 

The automated CIDSS makes It easier for me 
to make case assignments. 

The automated CIDSS makes It easier for me 
to spot potential problems In meeting 
deadi Ines. 

The automated CIDSS makes It easier for me 
to review cases currently being Investigated. 

The automated CIDSS makes It easl.sr for me 
to Identify shortcomings In quality of case 
recording. 

The automated CIDSS makes It easier for me 
to Identify shortcomings In quality of 
Investigations. 
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2. When you have to consult with a CPS specialist concerning a case they are Investigating, or 
have Investigated, the CIDSS record makes It harder or easier to: 

MUCH SOMEWHAT ABOUT SOMEWHAT "MUCH 
HARDER HARDER THE SANE EASIER EASIER 

Identify the major or Important aspects 

of the case. 1 2 3 4 5 

Why? 



Discuss the case with the Investigator 
Why? 



Communicate your concerns to the 

Investigator 12 3 4 5 

Why? 



3. IF YOU HAVE A CPS SPECIALIST WHO'S ;NLY EXPERIENCE WITH CASE INVESTIGATION RECORDING IS THE 
CIDSS SYSTEM: 

Did the CIDSS make It harder or easier for you to train the specialist? 

Harder Easier 
111 121 [31 141 151 

Why? 



Did the CIDSS make It harder or easier for you to supervise the specialist? 

Harder Easier 
til 121 . ^ 1 141 I 5 i 

Why? 
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COMMENTS 



One of the most Important uses of a pilot test Is the opportunity for hindsight. 
Please list below some things you feel could have been done better. 

We are especially Interested In: 

1. What additional Information, support or training would have been helpful? 



2. How would you change the automated CIDSS? Why? 
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COMMENTS 



3* Any other comments you have* Add pages If you need more space* When you have 
finished^ please fold In half with return address showing and staple closed* 
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John Theiss 
Research Design SpectaUst 
Technical Resources Section 
Research and Evaluation Division 
State Office, 232-E 
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TRAINING EVALUATION 



GENERAL INSTRUCTIONS: 



o Pkasc answer every question. If you like, you may discuss the training, the 
workbook, or training evaluation questions in the comment section. 

o We want to know your honest opinions. Please do not put your name on the 
questionnaire. 

o The identification number on this page helps us monitor the returns without 
placing your name on the evaluation or in the computer files. 



THANK YOU! 



G-24 



TRAINING FEEDBACK SECTION 



1. The Instructor demonstrated a genuine interest in this material. 

ll] DEFINITELY YES 

[2] YES 

[3] NOT SURE 

14) NO 

[5] DEFINITELY NO 

2. The instructor presented the material coherently, emphasizing major points and 
making relationships clear. • j f " «uu 

[11 DEFINITELY YES 

[2] YES 

[3] NOT SURE 

[4] NO 

[5] DEFINITELY NO 

3. The training adequately prepared me to use the CIDSS workbook. 

[1] DEFINITELY YES 

[2] YES 

[3] NOT SURE 

[4] NO 

[5] DEFINITELY NO 

4. What part(s) of the workshop will be most HELPFUL to you in doing your job? 



5. 



What parl(s) of the workshop will be LEAST helpful in doing your job? 



6. What suggestions do you have for IMPROVING the training? 
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II. OVERALL ASSESSMENT 



At the beginning of the training, you expressed some opinions concerning your 
expectations about the CIDSS Workbook. The following questions will tell us how the 
training affected your expectations. Please check the number of the statement that 
best describes your opinions. 

1. The CIDSS Workbook Is: 

[1] MUCH BETTER THAN I EXPECTED 

[2] SOMEWHAT BETTER THAN I EXPECTED 

[3] ABOUT WHAT I EXPECTED 

[4] SOMEWHAT WORSE THAN I EXPECTED 

[5] MUCH WORSE THAN I EXPECTED 

2. Do you think that THIS workbook is a GOOD IDEA for Child Protective Services? 

[1] DEFINITELY NO 

[2] PROBABLY NO 

[3] NOT SURE 

:4] PROBABLY YES 

[5] DEFINITELY YES 

I. Now that you are familiar vith the CIDSS Workbook, how do you think the 
workbook will change your job and the work you do? A number of possible 
changes are listed below. Please indicate your level of agreement or disagreement 
with each statement by checking the appropriate number on the seven-point scale 



THE CIDSS WORKEOOK WILL: 



1. Increase my workload 

2. Increase the freedom I ha^ 

3. Improve the quality of wo 

4. Make it more difficult to i 

5. Make it difficult to do a g( 

6. Make my work more chall< 

7. Make my work more frusti 

8. Decrease the discretion I c; 

9. Increase my ability to get > 

10. Make it easier to keep up 

1 1. Make my iob more interest 



• [I] 


[2] 


[3] 


[4] 


[5] 


[6] 


[7] 


• [1] 


[2] 


[3] 


[4] 


[5] 


[6] 


[7] 


.. [1] 


[2] 


[3] 


[4] 


[5] 


[6] 


[7] 


... [1] 


[2] 


[3] 


[4] 


[5] 


[6j 


[7] 


.. [1] 


[2] 


[3] 


[4] 


[5] 


[6] 


[7] 


... [1] 


[2] 


[3] 


[4] 


[5] 


[6] 


[7] 


[1] 


[2] 


[3] 


[4] 


[5] 


[6] 


[7] 


3. [1] 


[2] 


[3] 


[4] 


[5] 


[6] 


[7] 


... [1] 


[2] 


r3] 


[4] 


[5] 


[6] 


[7] 


... [1] 


[2] 


[:•] 


[4] 


[5] 


[6] 


[7] 


.. [1] 


12] 


[3] 


[4] 


[5] 


[6] 


[7] 
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III. SPECIFIC CONCERNS 



rj^r^"",?- 'f*" ""'i*'''' '"'^ "8"'^* statement or observations about the 
implemented" ' ^'^^^ workbook, and/or the way it is being 



implemented. 
POSITIVE 



NEGATIVE: 



G^27 20'^ 



IV. MATCHING THE MODEL AND THE WORKBOOK 

The CIDSS system is based on the investigation model presented in the training The 
workbcok was planned to correspond to the model. Several major sections of the model 
and the workbook are listed below. Please put an 'X' in every box that represents a 
match between the model and the workbook. 



WORKBOOK SECTIONS 

Allegations 

Evaluation of Children 

Description of Injuries 

Explanation of Allegation (adult) 
Family Ability to Protect 

Community Resources Used/Needed 
Supervisor Review 

MODEL 

EVENT: 
What happened 

EXTENT: 
How serious 

DYNAMICS: 
Environmeat 



RESOURCE 
Availability 
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USING THE CIDSS WORKBOOK 



The two paragraphs below include information from an intake and an initial 
investigation contact Please read this information and enter it in the appropriate 
places in the attached workbook. pp'vi^rimc 

On March 7, 1985 you were assigned the following Priority I abuse 
investigation. Pete S. arrived at school severely bruised with 
blackened eyes and a bloody ear. The school nurse alleges that Mr 
Riser abused Pete. The intake report contained Mr. Riser's work 
phone number. You immediately called him. 

Mr. Riser reported that he was only trying to teach his step-son, 
Pete, the importance of success in academic and sports activities. He 
said that he expects his son to excel in everything and punishes him 
severely When he docs not He repeatedly emphasized that he did 
not hit Pete and therefore does not abuse him. 
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This completes the training evaluation, wc would appreciate any ad 'itional comments on the 
training and/or the workbook. Please write them in the space below. When you have 
finished: 

0 Fold the training evaluation in half with the return address showing. 
0 Place you training evaluation form in the box by the door. 



COMMENTS: 
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TO: JEFFREY M. ANDERSON 

ORGANIZATION DEVELOPMENT DIVISION 
MAIL CODE 503-E 
STATE OFFICE 



ERIC 
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APPENDIX H 

Implementation Factors Survey 
for Evaluation of Automated CIDSS 
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CASE INVESTIGATION 
DECISION SUPPORT SYSTEM 
WORKBOOK 



CPS SPECIALISTS ASSESSMENT 
OF 

IMPLEMENTATION FACTORS 



Office of Strategic Management 
Research and Development 
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MONITORING THE IMPLEMENTATION OF 
THE CASE INVESTIGATION DECISION SUPPORT SYSTEM 



GENERAL IMSTRDCTIOHS 

o We want to know your honest opinions* Please do not write your name on 
the Implementation Factors Questionnaire* 

o The survey identification number on this page helps us monitor the 

questionnaire returns without placing your name on the questionnaire or 
in the computer files* 

o The questions ask you to circle one of several numbers that appear on a 
scale beside each statement* You are to choose the one number that best 
matches the description of how you feel about the statement* 

o You may write comments in the Problem Explanation sections and in the 
comment section at the end of the questionnaire. 



THANK YOU FOR PARTICIPATING 



ID 
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I. IHNOTATIOM FACTORS 



Ntflierous factors affeci: che success of an innovation* The ten factors listed 
balov are often linked vlth successful projects. Please indicate your level of 
agreement or disagreeaent vlth each statenent by circling the appropriate number 
on the seven point scale. 

AT THIS POINT IN TIME, HOW ACCTTRATELT 00 THESE STATEMENTS DESCRIBE THE CASE 
INVESTIGATION PKOJECT? 



/ 



1. The expected benefits for 

using the workbook are clear 



• • • 



2. The procedures for using 

the workbook are clear> 

3e The procedures for getting, 

completing, using, and storing 
the workbook are simple > . • • • 

4. The workbook configuration Is 
stable: content, format 
and procedures will not have 
to change much over time. • • • 



[1] 
[1] 



5 a There Is need for the workbook 
In my unit 



[1] 
[1] 



6 a Beyond meeting the need^ the 

workbook has obvious advantages 
for my unit » • • • • 



7. The effectiveness of the work- 
8. 



book Is readily observable 



• • • • • 



All components of rhe workbook 
are fully developed and 
readily available. ••••••• 



/ 4" ^■ 

^ ^ £^ / / / 



/ 



. (1] (2) 



[CT [3] [4] 
liif (3] (A] 



[3] 
[3] 



(1] [2] [4] 
(1] CV2f [3] [4] 



[5.J m: m 



[4] C5J f^J P'l 



[1] m [3] [4] rnhms 



9. Using the workbook, I can 
complete investigations as 
fast or faster than without 
the workbook 



• • • • • 



.[1] t^] [3] [4] |,5r, ,\|6J;, V 173 
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10. The workbook is a reliable 
decision support tool • • • 



^21" [3] [4] isi^'^'i^rar 
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FlOtLEM EXPLAHATION 



If you coded any responses in the darkened area please use this page to 
explain your response. IP TOU DO NOT AGREE WITH A STATEMENT PLEASE TELL 
US: 

A* What has happened to cause you to * disagree?* 

B« vhat can be done to change future entries to * agree 7* 

C« who or vhat organizational unit can resolve the problem? 

For example y a description for a disagree code for question 3: 

#3» (A) It is not easy to get new forms • The masters provided by State 
Office are not clear enough form making good copies* 

(B) New masters could be made and printed (In large quantities) at 
the State printshop In Austin* 

(C) I think the project director should take care of this. 



9 



II • am AHD SETTING FACTORS 



The succtM of tn innovation is linked to characteristics of the setting and 
the users as veil as characteristics of the innovation. The nine setting 
characteristics listed below are often linked with successful projects. Please 
indicate Tour level of agreement or disagreement with each statement by 
checking the appropriate number on the seven point scale. 

AT THIS POIHT IH TIME, HOW ACCTOATELT DO THESE STATEMEHTS DESCRIBE THE CASE 
INVESTIGATION PB0JECT7 



1« I am aware of the expected 

benefits from using the workbook. 

2. I have skills & knowledge 

needed to nse the workbook • • • 

3« It will be easy to retain 
experienced workbook users 
in my unit* ••••••••••4 

4. I feel motivated to give 

(the workbook a fair trial. . . . 

5. My beliefs and values 
make it easy for me to 
accept the workbook as 
legitimate practice. . o . . . . 

6. Facilities, equipment, and 
supplies are available to 
support nhe use of the 
workbook. 

7 . Leaders at all 1 evel s 

endorse the workbook 

8. I car identify and address 
factors that hinder my 

use of the workbook 



4' 



/// / 



4^ 



■tn [M" [3] [4] 

[1] [8f [3] [4] 



.[1] 
[1] 



[3] 

[of [3] 




[1] \zi [3] [4] ::-cSi?i 



[1] [3] [4] ■^^§^M0 



[2] [4] kMRMM 



[1] [3] [4] 



9. Policies and Procedures are In 
place to guide the use & security 
of the workbook [1] 



[VI [3] [4] '^t^ffM: --Vf. 
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PBOBLEH EZ?LA1IATI01I 



If you cod«d any rMpoosts in the darkened area please use this page to 
explain your response* IF TOO DO NOT AGREE WITH THE STATEMENT PLEASE TELL 
US: 

A* vhat has happened to cause you to 'disagree?* 

B« vhat can be done to change future entries to 'agree?* 

C« vho or vhat organizational unit can resolve the problem? 

A potential description for a disagree code for question 3: 

#3. (A) I an new to this unit. I did not receive the training (neither 
did three other CPS specialists). 

(B) Follow-up training is needed, or an orientation to the workbook 
needs to occur during the first weeks with the unit. 

(C) Either Snaff Development or the Project Director could provide 
training/orientation* 
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This completes the questionnaire. We would appreciate any additional 
comments you would like to malce. Pleaae write them In the space below. 
When you have finished, please: 



o Fold the questionnaire in half ^so the return address Is showing) 
and tape or staple the questionnaire shut*. 

o Return the completed questionnaire through agency mall. 

Thank you very much for participating. 

COMMENTS: 



er|c ""^ ^^'^ 



AGENCY V«AIL 



TO: JEFFREY M. ANDERSON 

OFFICE OF STRATEGIC MANAGEMENT, 
RESEARCH AND DEVELOPMENT 
STATE OFFICE M.C. 231-E 



■■; ' - - 13 1986 l :i 



r 
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CASE INVESTIGATION 
DECISION SUPPORT SYSTEM 
WORKBOOZ 



CPS SUPERVISOR'S ASSESSMENT 

OF 

IMPLEMENTATION FACTORS 



OFFICE OF RESEARCH 
DEMONSTRATION AND EVALUATION 
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Monioixsc m nmniDnATioi or 

THE CASI imsnCATIOV D7CISI0R SUPPOIT SYSTEM 



CEMIIAL nSTKR^xiOn 

o Vm vmat to kaov yout homnt opialou. F1m«« do hoc wttf your nm% on 
thm Liipl«Mittatloa Factors QaostiomiAlrt. 

0 Tho •ttTfoy idoitfifieaclott aoabor ov chit p«g« h«lpo us aoolcor cho 

quotciomiairt rocunis vlcliouc pladnt your lunio on ch« qutsclonnairc or 
la eho eosfmeor filos* 

o Tho quosciooa ask you Co drclo ooo of tOToral ombsrs that appaar oa a 
•cala baaida aaeh ocacaaant* Tou ara Co ehooaa cha ooa auabar that baat 
sacchaa cha daacrlpcioo of hov you faal abouc cha scacaaaiit* 

o Tou My vrlca coMants la cha Problan Ezplaisacloa aacclona and in cha 
LP— a lit MCCioa ac cha and of cha quaacionaalra* 



THANK YOU FOR PARTICIPATING 



ID 
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X. nwfmw PACiois 



VvMrote factors tfftet thm Mcetss of ma ioaovaclott* Tbm fn faecora listod 
Mov «ro of cm UidM4 vtch MCCMoful projoeta. PImm lodieato your IotoI of 
atroMMt or dioagraMo n t wleh aach atataaaat hj elrellat tha appropriaca maibar 
on tha aairaa poiat acala. 

Ai nzs POUT n tm, bov accoiaxilt do THBSI STAISSVn 

DISClIll TH CASl DIflSTIGmoV PIOJBCTT 



1. 



2. 



3. 



4. 



5. 



6. 



8. 



9, 



Tha axpaecad banafita for 
ualog tha workbook ara daar 

Tha procaduraa for aalng 
cha workbook ara daar*. • • 



Tha procadaraa for gactlatt 
coaqplaclst, oalat* m4 acorlag 
cha workbook ara al«pla> • • • 



Tha workbook eoaflfttration it 
scabla: eoacanc, forvac, 
and procaduraa will not hav« 
CO chanta auch OTar tlaa. • • « 

Thara la naad ior tha workbook 
la arf ualc 



Sayood Maeiog cha aaad, cha 
workbook haa ob^loua advanca^aa 
for mj oalc 



7. Tha affacclTasaaa of cha work* 



book la raadlly obaarrabla. 



All coapoaanta of cha workbook 
ara fully daralopad aad 
raadlly aTallabla« 



Oaiot tha workbook, ay tCaff 
can co«placa inr^sCigaCloaa 
aa faac or faacar Chan wlchouC 
Cha workbook 



10* Tha workbook la a raliabla 
daeialon support cooITTT 



/ 



// ^ ^ ^ 



4^ 
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(3) 


[4] 


(11 


(21 
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[4] 


(11 


(21 


(31 


[4] 


(1) 


(21 


[3] 


[4] 


(1) 


(21 


(3] 


[4] 


(11 


(21 


(3] 


[4] 


(11 


(?1 


(3] 


[4] 




(2) 


(31 


[4] 


> 

(1) 


(21 


(31 


[4] 


(1] 


(21 


(3) 


(^1 



4^ 



4* 



:5:.^:>YV.:.' 
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nOBLEM EZFLAimOH 



If 70a co4«d nj nspouts la ch« darkaiitd mtmm pl«M« m« this pagt to 
uplala your r«spotis«« IF TOD DO >0T A61ZE WITH k STAXEMEST PLEASE TELL 
US: 

VhAC hM9 lUpp«Md CO CAMt JOO CO 'diMgtM?* 

B« whmc can b« doii« Co chaag* fucurt tncritt Co *«gree?* 

C« vho or whac organlxACioul onlc out t%9olrm ch« problea? 

Por tsnpla^ a d«serlptloa for « dlsagrM cod« for quae clou 3: 

#3* (A) Ic Is HOC M«7 to g«c mv forms. Th« assctrs provided by State 
Office are act clear enough font aaklag good copies* 

(B) New aascers could be aade and princed (in large quancities) at 
the Scace princshop ia Aascin* 

(C) I chink Che pro. sec direccor should take care of this. 
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II. USE! AID SErrUW FACTOIS 

Thm sttccrjs of «a InnoTmcloa la linked eo elmrmce trite let of cht ttctlsig tnd 
cht ottrt tt vtll at eluurteetrltclct of cht limoTtcloii« Tht idm ttcclng 
ehtrtcctrltclct lltctd btlov trt ofttn Ilnktd vlch tucctttftil projtctt. Plette 
ladlctct your Itvtl of tgrttmtitt or dlttgreeatnc irlch ttch tctceaenc by 
chtcklng Cht tpproprltct nuabtr on tht ttvtn polnc sctlt* 

AT THIS roiHT IH TDK, HOW ACCnXAIELT DO THESE STATEMENTS DESCtlBE THE CASE 
nVESTIGATION PROJECT? 



• / /// f 

/ f f ^/ / ■ / / 



/ 



1. 


Scaff art avart of tht txptcctd 
btntflct froa utlng Cht vorkbook. 


.(!] 


[2] 


[3] 


(M 


2. 


SCa:!£ havt skills i knowltdgt 
netdtd Co uat Cht workbook • • • 


• [1] 


[2] 


[3] 


(*] 


3. 


Ic vUl bt taty Co rtc«lxi 

tzptrltnctd workbook uttrs 

la ay unlc •••••••••••• 


•til 


[21 


[3] 




4. 


Scaff ptrctlvt tht nttd for 
using tht workbook 


[1] 


(21 


[3] 


[*] 


5. 


Staff are aotlvactd to give 

tht workbook a fair trial .... 


. [1] 


[2] 


[3] 


[*] 


6. 


Staff *s btlltfs and valuta 
aakt Ic tasy for thea to 
acctpc tht workbook as 
ItgltlaaCt pracclct 


. (1] 


[2] 


[3] 




7. 


Fadlltlts, tqulpatnCy and 
•upplltt art avail ablt to 
support Cht ust of tht 
workbook* •••••••••••• 


.[l] 


[2] 


[3] 


[4] 


8. 


Ltadtrs ac all Itvtls 

tndorst tht workbook 


.[1] 


[2] 


[3] 




9. 


Polldts and Procedures are In 











place to guide the ude & security 
of the workbook 



[1] [2] [3] [4] 
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PioiuQf n?LAiimoi 

If 7M eodtd A0f rMpOMM 1a th* dmtkmmA AtM pImaa m« this p«s« to 
tzplain yoar r«ApOMA« 17 TOO DO HOT IGIEK WITH k STATEMEHT PLEASE TELL 
US: 

vhAt hmt hsppAMd to emutt you to *dltmgr«e?* 

B. what cum b« dOM to ehaaSA future tiitrltt to Ugr««7* 

C« who or what orgmiilMtloiMl unit can r«MlT« tha proUoi? 

▲ pot«9tlal dMcrlptlon for a dl«<rrM eo4« for quattloa 3: 

#3. (A) I ha^o throo ocv CPS Spodallsta who did not receive the 
trelnliig* 

(B) Follow-up tralslug le needed, or ma orleatatloa to the workbook 
needs to occur during the first weeks with the ua t« 

(C) Either Staff DeYelopseat or the Project Director could 
provide tralnlng/orleatatlon* 
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This completes the questionnaire • We would appreciate any addlciooal 
coments you would like to make. Please write them la the space below. 
When you have finished, please: 



o Fold tha quaatloimaira la half (so the return sddress is shoving) 
and tape or staple the questionnaire shut. 

o latura the covpletedi quaatloaaaire through agency mall. 

Thank you very much for partlcliatlng. 



COMMENTS: 
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MGBKt MilL 



TO: JEFFBET M. AHDBISOH 

OKAHIZAXIOll DSVELORODrr DIVISIOH 
TEXAS DEPAIXHEBT OF HOMAM SEXVICZS 

MAIL CODE 503-B > 
A0STX>, TISAS 
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GLOSSARY 



Automated MAPPER Intake (AMI) System — a mainframe 
computer application that replaced the Prompted 
Intake System. AMI allows for documentation of an 
intake report and electronic transfer of the intake 
report to another user, while simultaneously pro- 
ducing management information reports from the 
intake report data. 

CIDSS — Case Investigation Decision Support System, 
Two-part product developed by the Case Decision 
Project. Part I is a printed workbook that leads 
CPS specialists through a standardized method of 
handling CPS cases. Part II, automated CIDSS, is a 
software version of the workbook. Data from the 
automated CIDSS can be aggregated and reported in 
ways that aid management of CPS and decisions on 
cases . 

CPS Automation Plan — three-stage automation of DHS's 
child protective services, illustrated in figure 
I-l. 

CPS specialists — caseworkers who specialize in child 
protective services . 

DHS — Texas Department of Human Services. 

DSS — Decision Support System; a computer system that 
takes information relevant to a certain decision- 
making process, analyzes the information, and 
configures it in ways that allow the decision to be 
made more quickly and/or more accurately. 

Innovation Factors — Innate characteristics of an 
innovation that affect its acceptance. 

MAPPER — Maintaining, Preparing, and Producing Execu- 
tive Reports; a computer language. 
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MIS — Management Information System; a computer system 
in which information needed for management of a task 
or operation is collected and displayed for use by 
persons responsible for the completion and/or con- 
trol of that task or operation. 

Model Exercise — Post-training test of understanding 
the RIF/RAF Model* 

Prompted Intake System — an application for micro- 
computers that assists the CPS intake worker in 
documenting an intake report. The system carries 
the user through a series of on-screen prompts to 
ensure that all relevant information is obtained. 
Replaced by Automated MAPPER Intake (AMI) System. 

PSFC — Protective Services for Children Branch in DHS. 



RIF/RAF Model — Risk Intensity Factors/Resource Avail- 
ability Factors? a conceptual model of the decision 
process in CPS cases* 

User and Setting Factors — factors outside an innova- 
tion itself that influence its acceptance. 

Workbook — see CIDSS. 

Workbook Exercise — Post-training test of proficiency 
at using the CIDSS workbook. 
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